Flow-storm Debugger

Flow-storm is an omniscient time travel debugger designed for Clojure and ClojureScript. It allows developers to inspect and navigate through execution history. There are two primary ways to use it:

1. With ClojureStorm (Recommended): Swap your standard Clojure compiler for ClojureStorm during development. This automatically instruments your code so you don't have to manually manage instrumentation.
2. Vanilla FlowStorm: Add FlowStorm to your development classpath and instrument code by re-evaluating forms manually.

High-frequency functions (like mouse movement handlers) can generate excessive traces and exhaust heap memory. You can limit function calls per thread using the flowstorm.threadFnCallLimits JVM property.

Usage: Set the property in the format namespace/function:limit. When a function reaches its limit, FlowStorm stops recording it and all functions below it in the callstack.

Example JVM Option:

-Dflowstorm.threadFnCallLimits=org.my-app/fn1:2,org.my-app/fn2:4

Dynamic Modification via REPL: You can manage these limits at runtime using the following functions from flow-storm.runtime.indexes.api/:

• add-fn-call-limit
• rm-fn-call-limit
• get-fn-call-limits

ClojureStorm and ClojureScriptStorm are specialized development compilers. They are forks of the official Clojure and ClojureScript compilers, respectively, enhanced with automatic instrumentation.

Usage Pattern: Swap the official compiler for the Storm version during development (using deps CLI aliases or lein profiles). This ensures that instrumentation is applied automatically to your code as you work, while allowing you to use the standard, uninstrumented compiler for production and testing environments.

Events allow the Runtime to communicate information to the Debugger. All possible events are defined in flow-storm.runtime.events.

Event Lifecycle:

1. Buffering: If no debugger is subscribed, events accumulate in flow-storm.runtime.events/pending-events. They are dispatched as soon as a subscription occurs.
2. Queueing: On the debugger side, events accumulate in flow-storm.debugger.events-queue.
3. Dispatching: A specialized thread dispatches events. Most are processed by flow-storm.debugger.events-processor/process-event.
4. Custom Listeners: Any part of the debugger can listen to runtime events by adding a callback via flow-storm.debugger.events-queue/add-dispatch-fn.

FlowStorm is composed of three primary subsystems that work together to provide a debugging experience:

1. Instrumentation: Responsible for interleaving extra code into your program to trace its execution. It is independent of FlowStorm but requires a chosen instrumentation engine to function.
2. Runtime: Runs inside the debuggee process. It records traces emitted by the instrumentation system into registries (forms and threads) and manages the execution timeline.
3. Debugger: The user-facing component that provides a GUI (implemented in JavaFX) to explore recordings. It communicates with the runtime to fetch data and receive events.

For a visual representation of how these parts interact, refer to the high-level diagram.

To prevent expensive serialization and handle values that cannot leave the debuggee process (like infinite sequences), the flow-storm.runtime.debuggers-api does not return actual values. Instead, it returns flow-storm.types/ValueRef objects, which act as reified pointers to the values stored in the runtime registry.

To interact with these references, use:

• val-pprint: Prints a value into a string representation using the provided print-level and print-length.

To use FlowStorm, you must select one of the following instrumentation engines. FlowStorm sets up callbacks that these engines use when generating instrumentation:

• Hansel: A library that adds instrumentation by re-writing forms at macroexpansion time.
• ClojureStorm: A Clojure dev compiler that instruments by emitting extra bytecode.
• ClojurescriptStorm: A ClojureScript dev compiler that instruments by emitting extra javascript.

FlowStorm hooks into these via:

• flow-storm.tracer/[hansel-config | hook-clojure-storm | hook-clojurescript-storm]

Once instrumented, the code will trigger the following flow-storm.tracer functions during execution:

• flow-storm.tracer/trace-fn-call
• flow-storm.tracer/trace-fn-return
• flow-storm.tracer/trace-fn-unwind
• flow-storm.tracer/trace-expr-exec
• flow-storm.tracer/trace-bind

The Debugger is the GUI-based tool used to explore recordings.

• Entry Point: flow-storm.debugger.main/start-debugger.
• UI: Implemented as a JavaFX application using namespaces in flow-storm.debugger.ui.*.
• State Management: Uses a custom component system defined in flow-storm.state-management (similar to mount).
• Subsystems: Includes state, runtime-api, ui.main, events-queue, websocket-server, and repl.core.

In remote debugging mode, the websocket-server and repl client are active to facilitate communication with a remote runtime.

To prevent blocking the UI during heavy operations (like searching or collecting data from large timelines), FlowStorm uses an asynchronous Task system. Tasks can report progress and be interrupted.

Key Functions:

• From the Debugger: Use flow-storm.debugger.ui.tasks/submit-task to call a task function.
• On the Runtime: Use these utilities to implement interruptible loops:
  • flow-storm.runtime.debuggers-api/submit-batched-collect-interruptible-task: For collecting functionality that traverses the entire timeline.
  • flow-storm.runtime.debuggers-api/submit-find-interruptible-task: For looping through until the first match is found.