DOCSEARCHbyAlgolia
Submit Docs

ogx

repository·main·Indexed 27 days ago

https://github.com/ogx-ai/ogx

An open-source, agentic, and OpenAI-compatible API server designed as a model-agnostic replacement for the OpenAI API. It supports pluggable providers for various models (such as Llama, GPT, and Gemini) and infrastructure (including Ollama, vLLM, and cloud) to allow developers to build AI applications without changing application code.

Tokens
81.4K
Snippets
207
Records
471
Agent score
93%

What's inside ogx

  1. Overview of Remote Inference Adapters

    main
    OGX provides remote inference provider adapters to connect to external AI services. Supported providers include Anthropic, Azure OpenAI, AWS Bedrock, Cerebras, Databricks, Fireworks AI, Google Gemini, Groq, NVIDIA NIM, OpenAI, Ollama, and many others. The system also supports generic OpenAI-compatible endpoints via llama_openai_compat and generic passthrough via passthrough.

  2. Overview of Remote Provider Adapters

    main

    OGX uses remote provider adapters to connect OGX APIs to external services. These adapters allow you to use external inference engines, vector storage, file storage, and tool runtimes within the OGX ecosystem.

    Remote providers are categorized into four main types:

    • Inference: Connects to LLM providers (e.g., Anthropic, Azure, OpenAI, Groq, Ollama, NVIDIA NIM, etc.).
    • Vector IO: Connects to remote vector databases (e.g., Chroma, Elasticsearch, Milvus, PGVector, Qdrant, Weaviate).
    • Files: Connects to remote file storage (e.g., OpenAI, S3).
    • Tool Runtime: Connects to remote tool execution environments (e.g., Bing, Brave, MCP, Tavily, Wolfram Alpha).

  3. Overview of the ogx package

    main
    The ogx package is the main server implementation for the OGX ecosystem. It provides the runtime environment, built-in providers, a command-line interface (CLI), and all server-side logic. It is responsible for server startup, provider resolution, request routing, and storage.

  4. Overview of OGX capabilities

    main
    OGX is an open-source AI application server that composes inference, vector stores, file storage, tool calling, and agentic orchestration into a single process. It uses a pluggable provider architecture, allowing you to swap backends (e.g., moving from Ollama/FAISS locally to vLLM/PGVector in production) without changing your application code.

  5. Overview of OGX features and endpoints

    main

    OGX provides an agentic API server with the following capabilities:

    • Chat Completions & Embeddings: Standard /v1/chat/completions, /v1/completions, and /v1/embeddings endpoints.
    • Responses API: Server-side agentic orchestration including tool calling, MCP server integration, and built-in RAG (file search).
    • Vector Stores & Files: Managed document storage and search via /v1/vector_stores and /v1/files.
    • Batches: Offline batch processing via /v1/batches.
    • Skills: Management of versioned skill bundles via /v1alpha/skills.
    • Multi-SDK Support: Native support for Anthropic (/v1/messages) and Google GenAI (/v1alpha/interactions) APIs.

  6. Overview of OGX APIs

    main
    OGX provides a comprehensive set of APIs for building generative AI applications. All APIs follow OpenAI-compatible standards, allowing you to use existing OpenAI API clients and tools, migrate between providers seamlessly, and maintain consistent API contracts across different environments.

  7. Overview of OGX Client SDK Generation

    main

    The OGX client SDKs are generated using Stainless based on two source-of-truth configuration files. These files ensure that the SDKs remain in lock-step with the API specification.

    • openapi.yml: The OpenAPI specification for the OGX API.
    • config.yml: The Stainless configuration that instructs the generator on how to build the client SDKs.

    Both files are automatically generated by the scripts/run_openapi_generator.sh script to maintain synchronization between the FastAPI-based API definition and the Stainless configuration.

  8. Overview of OGX (Open GenAI Stack)

    main

    OGX is an open-source, vendor-neutral AI application server and Python library. It provides a stable API surface that implements major frontier lab APIs (OpenAI, Anthropic, Google) while allowing you to swap backend providers (inference engines, vector databases, safety backends) via configuration without changing your application code.

    Key capabilities include:

    • Multi-SDK Compatibility: Supports OpenAI-compatible endpoints, Anthropic Messages API (/v1/messages), and Google GenAI Interactions API (/v1alpha/interactions).
    • Server-Side Orchestration: Executes the inference-tool-inference loop on the server via the Responses API, centralizing security and state management.
    • Pluggable Architecture: Supports over 20 inference providers (e.g., vLLM, Ollama, Bedrock), 13 vector stores, and 7 safety providers.
    • Deployment Modes: Runs as an HTTP server for production or as a direct Python library for scripts and notebooks.

  9. Overview of supported RAG Benchmarks

    main

    The suite includes four primary benchmark types:

    BenchmarkTypePrimary MetricDescription
    BEIRRetrieval-onlynDCG@10Standard IR benchmarks (nfcorpus, scifact, arguana, fiqa, trec-covid)
    MultiHOP RAGEnd-to-end RAGEM / F1Multi-hop reasoning over news articles
    QReCCConversational RAGEM / F1Multi-turn conversational QA with scoped corpus per conversation
    Doc2DialDocument-grounded dialogueEM / F1Goal-oriented dialogues grounded in documents

  10. Overview of OGX Core Capabilities

    main

    OGX is an open-source platform that standardizes the building blocks for generative AI applications by aligning with the Open Responses specification. It provides a unified API layer that abstracts the complexity of different AI tools, vector databases, and model inference providers.

    Core capabilities include:

    • Inference: Consistent interface for running models locally or in the cloud.
    • Vector Stores: Building knowledge and agentic retrieval systems.
    • Agents: Creating intelligent agent flows using responses and conversations.
    • Tools and MCP: Integration with external services via direct tools or the Model Context Protocol (MCP).
    • Moderations: Built-in safety guardrails and content filtering.

  11. Understand the ogx directory structure

    main

    The ogx package is organized into the following functional modules:

    • core/: Server core including routing, resolution, storage, and the server itself.
    • providers/: All provider implementations (both inline and remote).
    • distributions/: Pre-built distribution configurations.
    • cli/: CLI commands such as ogx stack run, build, and configure.
    • models/: Model metadata and registries.
    • testing/: Test infrastructure, including an API recorder for record/replay.
    • telemetry/: OpenTelemetry integration.
    • env.py: Environment variable utilities.
    • log.py: Logging configuration.

  12. Supported Vector Store Providers for File Operations

    main

    OGX supports various vector store providers with different levels of file operations integration.

    Full File Operations Support

    These providers support complete integration including file upload, automatic processing, and search.

    Inline Providers (Single Node):

    • FAISS: Fast in-memory search, GPU acceleration.
    • SQLite-vec: Hybrid search, disk-based storage.
    • Milvus: High-performance, scalable indexing.

    Remote Providers (Hosted):

    • ChromaDB: Metadata filtering, persistent storage.
    • Qdrant: Payload filtering, advanced search.
    • Weaviate: GraphQL interface, schema management.
    • Postgres (PGVector): SQL integration, ACID compliance.

    Partial Support

    • Builtin: Provides core vector operations only, lacks full file operations integration.