Studyflow Documentation

Studyflow is a graphical notation used to describe and coordinate experiments (e.g., AI evaluation, behavioral research, clinical studies) and data analysis pipelines. It serves as a single source of truth that functions simultaneously as the protocol, plan, runnable code, analysis specification, provenance history, and final report.

Technically, a studyflow diagram is a valid BPMN 2.0 file extended with domain-specific elements. It can be saved as a YAML file or an image (SVG/PNG) with embedded metadata, allowing it to be opened in a browser, edited, or executed by a runner.

A studyflow is represented as a graph where nodes are events, tasks, gateways, and data. It utilizes two primary types of edges to define flow:

• Sequence flows (solid arrows): Define control flow (the order of operations). They may include a conditionExpression to determine if a branch is taken.
• Data associations (dotted arrows): Define data flow, indicating what a step reads or writes. These can describe bindings using the syntax slot = selection, where self binds a method's receiver and * appends values positionally.
• Message flows: Represent exchanges between two parties, using messageRef to name the Message and itemRef to name the ItemDefinition (structure).

The studyflow-modeler ecosystem consists of several key components:

• The Modeler: Used for authoring, editing, validating, importing, and exporting studyflows (packages/modeler).
• Browser Runtime: The browser skill that runs participant-facing steps, served at ./run/ (skills/browser).
• Local Runtime: The local skill that walks the diagram, records progress, and delegates elements to the appropriate skill (skills/local).
• Command Line Interface (CLI): Provides convert, validate, info, and run commands (packages/cli).
• Core: The shared object model handling YAML to BPMN conversion and element access (packages/core).
• Skills: BPMN extensions (vocabulary), runners for execution, and modeler modules (skills).

LLMs are treated as actors rather than step settings. A model is represented as a pool with actorType: llm and no internal process. Interaction occurs via message flows:

• Model Call: A task with a message flow to the model's pool and one back.
• Guardrail/Judge: A model call utilizing a strict prompt.
• Tool: A service task with an implementation.
• Agent Loop: An ad-hoc sub-process that terminates based on a completionCondition.

Every model exchange is recorded, including the model's digest, quantization level, sampling parameters, server version, and the full request/reply pair.

Pools communicate exclusively via message flows. The run carries each message between pools:

• Send Task: Sends its data inputs.
• Receive Task: Waits for the next message.
• Catch Event: Waits for a specific message.
• Throw/End Event: Sends a message.
• Boundary Event: A message at a boundary event terminates the step it is attached to.
• Event-based Gateway: Waits for the first arriving message to determine the path.

Message flows are defined by a messageRef pointing to a Message object, which in turn references an ItemDefinition via itemRef. The structureRef of the item definition is the technical token recognized by the runner (e.g., behaverse:Trial, agentic:Request).

In a studyflow, step transitions are governed by boundary events:

• Failure Handling: If a step fails and has an error boundary event, the run continues from that event, and the failure is recorded without failing the entire run. If no error boundary event exists, the failure ends the run.
• Sub-process Errors: An error end event inside a sub-process triggers that sub-process's error boundary event, allowing the participant to leave the loop rather than stopping the study.
• Conditional Boundaries: A step takes a specific path out if a conditional boundary event's condition (written as a conditionExpression) is met after the step's result is recorded. Placeholders can be used to reference step results (e.g., {Play.failedTrialRate} > 0.2).

To build a study, use the Studyflow Modeler (an open-source, browser-based tool).

1. Design the Flow: Use standard BPMN elements such as tasks, events, gateways, and flows.
2. Add Domain-Specific Elements: Incorporate specialized elements like questionnaires, cognitive tasks, brain imaging sessions, or Python data analysis functions.
3. Define Metadata: Describe each element by attaching metadata, such as the specific script to run, the data file to read, or the figure to produce.
4. Save/Export: The result is saved as a YAML file or an image with embedded metadata.

You can add a custom vocabulary as an extension set. New types must extend an existing type and add their own attributes to maintain BPMN compatibility.

What declaring a type provides:

• A place in the palette, a specific shape, and an icon.
• Fields in the inspector.
• Storage and retrieval across all supported file formats.

What declaring a type DOES NOT provide:

• A dedicated screen or call in the runner.
• Pre-flight validation for author-typed values.
• Automatic execution logic during a run.

To make a new type functional during a run, you must teach the runner how to render or execute it.

For local runs, you can execute new elements without modifying the main runner by creating a 'skill'. A skill pairs a schema extension with a partial runner executable.

Implementation Details:

• Executable: Create a file at skills/<name>/local.py or any executable named studyflow-<name> available on the system PATH.
• Execution Flow: The partial runner claims a subset of the diagram's elements. It receives a JSON digest of the plan and the current state, and it must output the updated state.
• Contract: The full technical contract is defined in the local runtime's SKILL.md. Examples can be found in skills/python and skills/reachy.

Studyflow diagrams can be executed by 'runners' across various environments, including web browsers, local Python environments, HPC, or containers.

When a diagram is run via the Runner app, the output is a git repository containing:

• Checkpoints
• Figures
• Logs
• All intermediate files

The provenance of the run (what ran, when, and by whom) is stored directly within the diagram file and reflected in the git commits of the output repository.