DOCSEARCHbyAlgolia
Submit Docs

Priompt Documentation

repository·main·Indexed 25 days ago

https://github.com/anysphere/priompt

A JSX-based prompting library that uses a priority-based system to intelligently manage LLM context windows based on token limits. It includes components for chat messages (System, User, Assistant), image integration, and tool definition via Zod schemas (ZTools, ZFunction). The ecosystem also provides @anysphere/priompt-preview for prompt visualization and @anysphere/tiktoken-node, a specialized tiktoken fork with asynchronous tokenization and special token support.

Tokens
7.5K
Snippets
17
Records
71
Agent score
83%

What's inside priompt

  1. Overview of @anysphere/tiktoken-node

    main

    @anysphere/tiktoken-node is a specialized fork of tiktoken designed with specific enhancements for advanced use cases. Key features include:

    • Special Token Support: Added support for handling special tokens.
    • Asynchronous Tokenization: Support for running tokenization asynchronously, offloading computation to a different thread to avoid blocking the main event loop.

  7. Use Priorities to Manage Context Window

    main

    Priompt uses priorities to decide which parts of a JSX tree to include in the context window.

    • Absolute Priority: Use the p prop on a <scope>.
    • Relative Priority: Use the prel prop on a <scope>. A higher priority means the child is more important to include.
    • Default Behavior: If no priority is specified, a child is included if and only if its parent is included.

    Example of prioritizing recent history over older history:

    {props.history.map((m, i) => (
  <scope prel={-(props.history.length - i)}>
    {m.case === "user" ? (
      <UserMessage>{m.message}</UserMessage>
    ) : (
      <AssistantMessage>{m.message}</AssistantMessage>
    )}
  </scope>
))}
    function ExamplePrompt(
  props: PromptProps<{
    name: string,
    message: string,
    history: { case: "user" | "assistant", message: string }[],
  }>
): PromptElement {
  const capitalizedName = props.name[0].toUpperCase() + props.name.slice(1);
  return (
    <>
      <SystemMessage>
        The user's name is {capitalizedName}. Please respond to them kindly.
      </SystemMessage>
      {props.history.map((m, i) => (
        <scope prel={-(props.history.length - i)}>
          {m.case === "user" ? (
            <UserMessage>{m.message}</UserMessage>
          ) : (
            <AssistantMessage>{m.message}</AssistantMessage>
          )}
        </scope>
      ))}
      <UserMessage>{props.message}</UserMessage>
      <empty tokens={1000} />
    </>
  );
}

  8. Install Priompt and Priompt Preview

    main

    Install the core @anysphere/priompt library and the @anysphere/priompt-preview development dependency using your preferred package manager.

    npm install @anysphere/priompt && npm install -D @anysphere/priompt-preview

# or

yarn add @anysphere/priompt && yarn add --dev @anysphere/priompt-preview

# or

pnpm add @anysphere/priompt && pnpm add -D @anysphere/priompt-preview

  9. Run the Priompt examples

    main

    To run the demonstration of priompt and priompt-preview, follow these steps:

    1. Initialize the environment: Run the initialization script from the parent directory.
    2. Configure API Key: Set your OpenAI key in a .env file.
    3. Start the server: Run pnpm priompt in one terminal.
    4. Start the previewer: Run pnpm watch in a second terminal.
    5. Test the endpoint: Use curl to send a message to the local server.

    You can view the prompt in the priompt-preview interface at http://localhost:6284.

  12. Understand the prompt rendering process

    main

    Priompt renders prompts by traversing a tree of PromptElement nodes. The rendering process involves:

    1. Normalization: Merging strings and preparing nodes.
    2. Priority Computation: Determining available priority levels.
    3. Search: Using algorithms like renderBackwardsLinearSearch or binary search to find the highest priority level that fits within the tokenLimit.
    4. Token Counting: Performing an exact token count after the initial estimation to ensure accuracy, especially when tokens span scope boundaries.