Pocketflow Framework

repository·main·Indexed 20 days ago

https://github.com/osly-ai/pocket-flow-framework

A lightweight, vendor-agnostic TypeScript framework for building LLM-powered workflows using nested directed graphs. It provides primitives for branching, cycles, retries, and parallel batch execution through abstractions like Node, Action, Flow, and Shared State. The framework includes specialized components such as RetryNode for resilience, BatchFlow for concurrent processing, and AsyncNode/AsyncFlow for asynchronous operations, along with a pocket-cli for scaffolding components.

Tokens
38.1K
Snippets
85
Records
106
Agent score
70%

What's inside pocketflow

  1. Overview of the Pocket Flow Framework

    main

    Pocket Flow Framework is a modular, vendor-agnostic framework designed for building robust LLM-powered applications. It provides the foundational abstractions required to implement intelligent agents, task decomposition, Retrieval Augmented Generation (RAG), and complex multi-step processing workflows.

    Key use cases include:

    • Building Intelligent Agents
    • Implementing Task Decomposition
    • Developing RAG pipelines
    • Managing complex multi-step logic
  2. What is a Node and how to implement it

    main

    A Node is the fundamental building block of a flow in the Pocketflow framework. To create a custom node, extend BaseNode and optionally override the following three lifecycle methods to separate concerns:

    1. prepAsync(sharedState): Use this for preprocessing data. It is a reliable step to query databases, read files, or serialize data. It returns a prepResult which is passed to the next step.
    2. execAsync(prepResult): This is the main execution step (e.g., an LLM call or remote API request).
      • Important: Do not write to sharedState here. Instead, gather necessary data in prepAsync and pass it via prepResult.
      • If retries are enabled, ensure this implementation is idempotent.
      • Returns an execResult passed to postAsync.
    3. postAsync(sharedState, prepResult, execResult): Use this to write results back to the sharedState and determine the next step in the flow. It must return a string representing the next Action (e.g., "default").

    You do not need to implement all three; for example, a node that only prepares data might only implement prepAsync.

    import { BaseNode } from "../src/pocket";
    
    export class MyCustomNode extends BaseNode {
      public async prepAsync(sharedState: any): Promise<any> {
        // Prepare data
        return "prepared data";
      }
    
      public async execAsync(prepResult: any): Promise<any> {
        // Main logic (e.g. LLM call)
        return "execution result";
      }
    
      public async postAsync(sharedState: any, prepResult: any, execResult: any): Promise<string> {
        // Write to state and return next action
        sharedState.result = execResult;
        return "default";
      }
    }
  3. Implement Nested Flows for complex workflows

    main
    A Nested Flow is a Flow that is treated as a single Node within a parent Flow. This pattern allows you to encapsulate complex, reusable logic into smaller, manageable units. For example, an 'Order Processing' Flow might contain a nested 'Payment Verification' Flow. This modularity makes large-scale workflow orchestration easier to design and maintain.
  4. How Shared State Management works

    main

    The framework maintains a sharedState object that is passed through every node in a flow. This object acts as a global context that nodes can read from and write to, allowing for data persistence and communication between disparate nodes in a workflow.

    // Each node can read/write to shared state
    async post(prepResult: any, execResult: any, sharedState: any): Promise<string> {
        sharedState.results.push(execResult);
        return "default";
    }
  5. How the Nested Directed Graph architecture works

    main

    Pocket Flow models LLM workflows using a Nested Directed Graph abstraction. This architecture allows developers to break complex problems into manageable, interconnected, and reusable components.

    The core components of this model are:

    • Nodes: The atomic units that handle specific LLM tasks or operations.
    • Actions: Labeled edges that connect nodes, enabling conditional logic and agent-like behaviors.
    • Flows: Orchestrators that manage the graph of nodes, handling execution sequences and task decomposition.
    • Nesting: A key feature where an entire flow can be encapsulated and treated as a single node within a larger, parent flow.
    • Batch Processing: Support for efficient handling of data-intensive tasks.
    • Async Support: Support for parallel execution to optimize performance.
  6. How Action-Based Transitions and branching work

    main

    Nodes in Pocket Flow do not connect via static links, but through action-based transitions. The post() method of a node returns a string (an 'action') which the framework uses to look up the next node in the sequence.

    To implement conditional branching or error handling, you register successors mapped to specific action strings using addSuccessor(node, action).

    // Define transitions based on actions
    nodeA.addSuccessor(nodeB, "success");
    nodeA.addSuccessor(errorNode, "error");
  7. Understand the four core primitives of Pocketflow

    main

    Pocketflow is built on four minimal primitives that allow you to express complex LLM workflows as Nested Directed Graphs:

    1. Node: The atomic unit of work. It follows a three-phase lifecycle (prepexecpost).
    2. Flow: A graph walker that traverses nodes based on labeled edges.
    3. Action: A string returned by a node's post() phase that determines which outgoing edge (and thus which successor node) the Flow should follow.
    4. Shared State: A plain JavaScript object passed to every node, used for inter-node communication and data persistence across the workflow.
  8. Implement Branching and Cycles in a Flow

    main

    You can implement complex logic like branching and loops by returning specific action strings from a node's post() method and mapping those strings to successors using addSuccessor.

    • Branching: Return different strings (e.g., "approved", "reject") based on logic in post() to route to different nodes.
    • Cycles: Create a loop by adding a successor that points back to a previously visited node in the graph.
    class QualityCheckNode extends BaseNode {
      async post(prepResult: any, execResult: any, sharedState: any) {
        if (execResult.score >= 0.8) return "approved";
        if (sharedState.retryCount < 3) return "retry";
        return "reject";
      }
    }
    
    const check = new QualityCheckNode();
    const publish = new PublishNode();
    const revise = new ReviseNode();
    const reject = new RejectNode();
    
    check.addSuccessor(publish, "approved");
    check.addSuccessor(revise, "retry");
    check.addSuccessor(reject, "reject");
    
    // Create a cycle: revise feeds back into check
    revise.addSuccessor(check, DEFAULT_ACTION);
  9. How Parallel Processing works with BatchFlow

    main

    To process multiple items in parallel, use the BatchFlow class.

    • In the prep phase, return an array of items from the sharedState that need processing.
    • In the post phase, the framework provides arrays of both prepResults and results (one for each item), allowing you to aggregate them back into the sharedState.

    This enables scalable, concurrent execution of tasks within a single flow structure.

    class BatchFlow extends Flow {
        async prep(sharedState: any): Promise<any[]> {
            return sharedState.items; // Return array of items to process
        }
    
        async post(prepResults: any[], results: any[], sharedState: any): Promise<string> {
            sharedState.results = results;
            return DEFAULT_ACTION;
        }
    }
  10. How Flows orchestrate Node execution

    main
    A Flow is the primary orchestration unit in the pocket.ts framework. It manages the execution of Nodes by interpreting the Actions returned by each node's postAsync() method. Instead of hardcoding a sequence, the Flow uses these Action strings to determine which Node should execute next, enabling dynamic, data-driven transitions.
  11. How Flows and Nodes compose (Nested Composition)

    main

    In Pocketflow, a Flow extends BaseNode. This is a critical design feature: it means a Flow can be treated as a single node within another Flow.

    You can build complex, hierarchical workflows by nesting sub-flows inside parent flows. This allows you to decompose monolithic graph definitions into small, well-tested, and reusable components.

  12. Implement the Map-Reduce paradigm in Pocketflow

    main

    The Map-Reduce paradigm allows you to process large inputs by splitting them into smaller chunks (the Map step) and then combining those results into a single output (the Reduce step).

    To implement this, you typically create two specialized BaseNode subclasses:

    1. Map Node:

      • prepAsync: Splits the large input (e.g., from sharedState) into an array of chunks.
      • execAsync: Processes each individual chunk (e.g., via an LLM call).
      • postAsync: Collects the results from all chunk executions and saves them into the sharedState for the next node.
    2. Reduce Node:

      • prepAsync: Retrieves the array of processed chunks from the sharedState.
      • execAsync: Performs a final operation (e.g., merging or summarizing) on the entire collection of results.
      • postAsync: Saves the final consolidated result into the sharedState.

    You connect them by calling mapNode.addSuccessor(reduceNode, DEFAULT_ACTION) and initializing a Flow with the map node.

    import { BaseNode, Flow, DEFAULT_ACTION } from "../src/pocket";
    
    // 1. Define the Map Node
    export class MapSummaries extends BaseNode {
      public async prepAsync(sharedState: any): Promise<string[]> {
        // Logic to split sharedState.text into chunks
        return chunks;
      }
    
      public async execAsync(chunk: string): Promise<string> {
        // Logic to process a single chunk
        return result;
      }
    
      public async postAsync(sharedState: any, prepResult: string[], execResultList: string[]): Promise<string> {
        sharedState.summaries = execResultList;
        return DEFAULT_ACTION;
      }
    }
    
    // 2. Define the Reduce Node
    export class ReduceSummaries extends BaseNode {
      public async prepAsync(sharedState: any): Promise<string[]> {
        return sharedState.summaries ?? [];
      }
    
      public async execAsync(summaries: string[]): Promise<string> {
        // Logic to combine all summaries
        return finalResult;
      }
    
      public async postAsync(sharedState: any, prepResult: string[], execResult: string): Promise<string> {
        sharedState.final_summary = execResult;
        return DEFAULT_ACTION;
      }
    }
    
    // 3. Wire them together
    const mapNode = new MapSummaries();
    const reduceNode = new ReduceSummaries();
    mapNode.addSuccessor(reduceNode, DEFAULT_ACTION);
    
    const flow = new Flow(mapNode);
    await flow.runAsync({ text: "large input..." });