DOCSEARCHbyAlgolia
Submit Docs

DBOS Documentation

website·Indexed 35 days ago

https://docs.dbos.dev

DBOS is a high-performance library for building reliable, durable workflows and applications on top of Postgres. It provides tools for observability, reproducibility, and fault tolerance, supporting multiple languages including Python, TypeScript, Go, and Java. Key features include durable transactions via datasources, reliable human-in-the-loop integration, parallel tool calls for AI agents, and a drop-in replacement path for Temporal users via DBOSify.

Tokens
277.5K
Snippets
1.3K
Records
1.6K
Agent score
62%

What's inside DBOS

  1. Introduction to DBOS Transactions and Datasources

    A DBOS transaction is a specialized workflow step that executes a function within a database transaction and records its return value. This mechanism guarantees that the transaction is executed exactly once, even during retries or failures. The DBOS datasource layer provides plugin packages for popular database libraries to ensure consistent transaction control, type safety, and correct runtime behavior across environments.

  2. Overview of the Conductor API

    Conductor is the HTTP control plane for managing durable workflows. It allows developers to programmatically register applications, tune settings, search and cancel workflows, inspect queues and schedules, read metrics/audit logs, and manage members, roles, and API keys. It serves as the backend for the DBOS console and the dbosctl CLI.

  3. Overview of S3Mirror for durable S3 data transfers

    S3Mirror is a DBOS-powered utility designed for performant, durable, and observable data transfers between S3 buckets. It utilizes a "fanout" pattern where a transfer is split into independent tasks processed via a DBOS queue. Each file transfer is wrapped in its own DBOS Workflow, ensuring that if the application crashes, the transfer resumes only for files that have not yet completed. On DBOS Cloud Pro, the application auto-scales by launching new VMs based on queue length.

  4. Overview of DBOS Conductor capabilities

    DBOS Conductor serves as the control plane for durable workflows in production. It provides several critical operational capabilities without being in the direct orchestration path (meaning applications continue to operate normally if the connection to Conductor is interrupted). Key features include:

    • High Availability: Automatically detects interrupted workflows and recovers them to healthy executors.
    • Observability: Dashboards for active/past workflows, queued tasks, real-time visualization, and a Prometheus-compatible metrics endpoint.
    • Management: Ability to pause, start, or restart workflows from specific steps via the dashboard or API.
    • Retention: Managed policies for how long workflow history is retained.
    • Autoscaling: Computes required executors based on queue utilization for tools like KEDA.
    • Programmatic Access: Management via an OpenAPI HTTP API and the dbosctl CLI.

  5. Overview of DBOS Cloud features

    DBOS Cloud is a serverless platform for durably executed applications. It provides:

    • Application hosting and autoscaling: Managed hosting that scales to millions of users, charging only for consumed CPU time.
    • Managed workflow recovery: Automatic recovery of workflows by another executor if a cloud executor is interrupted, crashes, or restarts.
    • Workflow and queue management: Dashboards to monitor active/past workflows and queued tasks, allowing users to cancel, resume, or fork executions.

  6. Overview of DBOS and Pydantic AI integration

    By wrapping a Pydantic AI agent in a DBOSAgent, you enable durable execution. This ensures that agents preserve progress across transient API failures, application errors, and restarts. DBOS runs in-process as a library, checkpointing standard Python functions to a database for durability.

    DBOSAgent automatically handles the following as DBOS workflows or steps:

    • Agent.run and Agent.run_sync (the main agent loop) are wrapped as DBOS workflows.
    • Model requests and MCP communication are wrapped as DBOS steps.

    Custom tool functions and event stream handlers are NOT automatically wrapped. Developers should:

    • Use @DBOS.step for functions involving non-determinism or I/O.
    • Skip the decorator for durability-unnecessary tasks (e.g., debug logging) to avoid checkpoint overhead.
    • Skip the decorator if the function needs to enqueue tasks or invoke other DBOS workflows (so it runs inside the main workflow).

  7. Overview of DBOS Transactions and Datasources

    DBOS Transactions are specialized steps for database access that execute as a single atomic database transaction. They commit both user-defined changes and a DBOS checkpoint simultaneously to ensure that each transaction is run exactly once. DBOS provides 'datasources' which wrap popular TypeScript database clients with this DBOS-aware transaction logic, maintaining the same interface as the underlying client library.

  8. Overview of DBOS Durable Workflows

    DBOS is an open-source library that provides high-performance durable workflows built on top of Postgres. It allows developers to write long-lived, reliable code that survives failures, crashes, and restarts without losing state or duplicating work. DBOS achieves this by checkpointing the program's state in Postgres; if a program fails, it automatically resumes from the last completed step upon restart. It requires no additional infrastructure beyond a Postgres database.

  9. Implement a durable AI research agent with DBOS

    DBOS allows the creation of reliable, durable AI agents by annotating workflows and steps. By implementing an agent as a durable workflow, it can recover from failures and resume research from the last successful step, preventing loss of work. This pattern is particularly useful for iterative agentic loops that involve multiple LLM calls and external API searches.

  10. Understand how DBOS workflow scheduling and execution works

    DBOS workflow schedules operate via an enqueueing mechanism rather than direct invocation. When a schedule fires, the workflow is enqueued by name. This architecture provides several benefits:

    1. Decoupled Execution: The process hosting the schedule does not need to have the workflow registered; name resolution occurs at dequeue time on a worker that possesses the function.
    2. Cross-Process/Language Support: Any process connected to the system database can drive schedules for workflows owned by other processes or different languages.
    3. Version Safety: Scheduled workflows always execute against the latest registered version of the owning application, preventing stale executors from picking up tasks after a new deployment.

  11. Understand DBOS cross-language serialization

    DBOS supports multiple languages (Python, TypeScript, Go, and Java). By default, each SDK uses a language-specific serialization format (e.g., pickle for Python, SuperJSON for TypeScript) that is not readable by other languages. To exchange data across different languages via workflows, messages, events, and streams, use the portable_json format. This format is a common data representation that all SDKs can read and write, and can be accessed directly from the database without DBOS code.

  12. Integrate DBOS durable workflows with Next.js

    Because Next.js may evaluate modules multiple times in different contexts, running DBOS directly inside the Next.js process is difficult. The recommended architecture is a two-process setup where both processes connect to the same Postgres database:

    1. Next.js App: Enqueues workflows using the DBOSClient from server actions.
    2. DBOS Worker: A standalone process that dequeues and executes the workflows.