Open Assistant API

Open Assistant API is an open-source, self-hosted AI assistant API that is compatible with the official OpenAI interface. This compatibility allows you to use the official OpenAI Client to build LLM applications.

Key features include:

• LLM Support: Supports various models via One API.
• RAG Engine: Supports the R2R RAG engine for advanced retrieval-augmented generation.
• Web Search: Built-in support for internet searching.
• Tools: Supports custom functions and extensible built-in tools.
• Streaming: Supports message streaming for a smoother user experience.
• Deployment: Can be deployed locally.

When APP_AUTH_ENABLE is active, the API uses Bearer tokens for user isolation:

1. Verification: Include the token in the header as Authorization: Bearer ***.
2. Admin Access: Administrative APIs require the token configured in APP_AUTH_ADMIN_TOKEN (default is admin).
3. Token Creation: When creating a token, you must provide the LLM base_url and api_key. The assistants created with that token will use these credentials to access the model.

Open Assistant API allows integrating various tools (following OpenAPI/Swagger specifications) to enhance assistant capabilities, such as web search or code execution.

Workflow

1. Create Tools: Define and create the tools first.
2. Integrate: Attach the created tools to an assistant.
3. Authentication: If a tool requires authentication, you can add the necessary credentials at runtime.

For implementation details, refer to the test cases in the repository: tests/tools/assistant_action_test.py and tests/tools/run_with_auth_action_test.py.

The repository is organized into several key directories:

• app/api: API controllers (v1, routes, dependencies).
• app/core: Core logic including document loaders, runners, and tools.
• app/models: Database models.
• app/providers: Core service providers (database, storage, celery, middleware, etc.).
• app/services: Business logic layer.
• app/schemas: Data models/schemas.
• app/tasks: Background tasks.
• config/: Configuration files for app, database, storage, LLM, and logging.
• migrations/: Database migration scripts.
• tests/: Unit and end-to-end tests.
• main.py: Entry point for the API.
• worker.py: Entry point for the Celery task scheduler.

The API follows OpenAPI/Swagger specifications, allowing you to integrate various tools into your assistants to connect them to the external world (e.g., code execution, accessing private data sources).

To use a tool:

1. Create the tool.
2. Associate the tool with an assistant.

For tools requiring authentication, you can provide authentication parameters at runtime. Refer to the API documentation for specific parameter formats.

See tests/tools/assistant_action_test.py and tests/tools/run_with_auth_action_test.py for implementation examples.

To meet SaaS deployment requirements, you can enable simple user isolation based on tokens by configuring APP_AUTH_ENABLE.

Authentication Details

• Method: Bearer token. Include Authorization: Bearer *** in your request headers.
• Admin Token: The token used for administrative tasks (like token management) is configured via APP_AUTH_ADMIN_TOKEN (defaults to admin).
• Token Creation: When creating a token, you must provide the base URL and API key of the target large model. The assistant created with that token will use those specific configurations.

When you modify the database models in app/models, you must update the database schema using Alembic:

1. Generate a new migration script based on model changes:

   alembic revision --autogenerate

2. Apply the migration to the database:

   alembic upgrade head

To handle runtime errors gracefully in your application, wrap components or layouts that require error monitoring with the ErrorBoundary component. You can provide a custom component via the FallbackComponent prop to define how the UI should look when an error is caught.

<ErrorBoundary FallbackComponent={CustomerError}>{props.children}</ErrorBoundary>

The easiest way to start Open Assistant API is using Docker Compose. Ensure Docker and Docker Compose are installed on your machine.

1. Open docker-compose.yml in the project root.
2. Configure your environment variables (see Configuration).
3. Run the following command:

docker compose up -d

The project uses Ruff and Black for code quality. Before committing code, use the following commands to ensure compliance with the project's style guide:

• Check for formatting issues: make lint
• Automatically format code: make format

To run the project locally, ensure you have Python >= 3.10 and Poetry installed. Follow these steps to prepare your environment:

1. Install Poetry:

   curl -sSL https://install.python-poetry.org | python3 -
   # or
   pip install poetry

2. Install Dependencies:

   poetry install --no-root

3. Configure Environment Variables: Copy the example configuration file to .env and populate the required keys:

   cp .env.example .env

   In the .env file, configure:

   • OPENAI_API_KEY: Your OpenAI API key.
   • BING_SUBSCRIPTION_KEY: Your Bing search subscription key.

4. Start Middleware (MySQL, Redis, MinIO):

   docker compose -f docker-compose.middleware.yml up -d