oai-compatible-copilot

repository·main·Indexed 20 days ago

https://github.com/johnnyz93/oai-compatible-copilot

A VS Code extension (v0.4.2) that integrates external OpenAI-compatible, Ollama, Anthropic, and Gemini API providers into GitHub Copilot Chat. It allows developers to use their own models and API keys via a visual configuration UI or JSON settings. The extension supports five API protocols (openai, openai-responses, ollama, anthropic, and gemini), custom HTTP headers, and provider-specific parameters through an 'extra' field to enable features like reasoning/thinking blocks.

Tokens
9.8K
Snippets
26
Records
40
Agent score
73%

What's inside oai-compatible-copilot

  1. Manage Multiple Providers and API Keys

    main

    The owned_by field in a model configuration (also known as provider or provide) is used to group provider-specific API keys. The extension stores these keys using the pattern oaicopilot.apiKey.<providerId_lowercase>.

    To configure keys for multiple providers:

    1. Open VS Code settings and define your models with different owned_by values.
    2. Open the Command Palette (Ctrl+Shift+P) and run OAICopilot: Set OAI Compatible Multi-Provider API Key to enter the keys for each provider.
    "oaicopilot.models": [
        {
            "id": "Qwen/Qwen3-Coder-480B-A35B-Instruct",
            "owned_by": "modelscope",
            "context_length": 256000,
            "max_tokens": 8192,
            "temperature": 0,
            "top_p": 1
        },
        {
            "id": "qwen3-coder",
            "owned_by": "iflow",
            "baseUrl": "https://apis.iflow.cn/v1",
            "context_length": 256000,
            "max_tokens": 8192,
            "temperature": 0,
            "top_p": 1
        }
    ]
  2. Create Multiple Configurations for the Same Model

    main

    You can define multiple configurations for the same model ID by using the configId field. This allows you to use the same base model with different parameters (e.g., enabling/disabling Chain of Thought) for different scenarios. Each configuration must have a unique configId.

    In the VS Code model selector, these will appear as distinct entries using the format modelId::configId.

    "oaicopilot.models": [
        {
            "id": "glm-4.6",
            "configId": "thinking",
            "owned_by": "zai",
            "temperature": 0.7,
            "top_p": 1,
            "thinking": {
                "type": "enabled"
            }
        },
        {
            "id": "glm-4.6",
            "configId": "no-thinking",
            "owned_by": "zai",
            "temperature": 0,
            "top_p": 1,
            "thinking": {
                "type": "disabled"
            }
        }
    ]
  3. Use the `extra` field for custom request body parameters

    main

    The extra field allows you to inject arbitrary parameters directly into the API request body. This is useful for accessing vendor-specific features or experimental parameters not covered by the standard configuration fields.

    Key behaviors:

    • Parameters in extra are merged into the request body.
    • They are added after standard parameters. If a conflict occurs, the value in extra takes precedence.
    • Supported across all apiMode values (openai, openai-responses, ollama, anthropic, gemini).
    • Values can be any valid JSON type (string, number, boolean, object, array).

    Common use cases:

    • OpenAI specific: seed, logprobs, top_logprobs, suffix, presence_penalty.
    • Vendor specific: Custom sampling methods, debugging flags, or Beta features.
    • Enabling Thinking/Reasoning: Using vendor-specific nested objects to enable chain-of-thought displays.
    // Example: Using extra for OpenAI, Ollama, and Anthropic specific settings
    "oaicopilot.models": [
        {
            "id": "custom-model",
            "owned_by": "openai",
            "extra": {
                "seed": 42,
                "logprobs": true,
                "top_logprobs": 5,
                "suffix": "###",
                "presence_penalty": 0.1
            }
        },
        {
            "id": "local-model",
            "owned_by": "ollama",
            "baseUrl": "http://localhost:11434",
            "apiMode": "ollama",
            "extra": {
                "keep_alive": "5m",
                "raw": true
            }
        },
        {
            "id": "claude-model",
            "owned_by": "anthropic",
            "baseUrl": "https://api.anthropic.com",
            "apiMode": "anthropic",
            "extra": {
                "service_tier": "standard_only"
            }
        }
    ]
  4. Configure models to show 'Thinking' blocks

    main

    You can enable model reasoning/thinking displays in Copilot by using provider-specific parameters within the extra field or specific reasoning fields.

    For OpenAI Responses: Set apiMode to openai-responses and use reasoning_effort along with a reasoning.summary object in extra.

    For Gemini: Set apiMode to gemini and use generationConfig.thinkingConfig.includeThoughts inside the extra field.

    // OpenAI Responses with detailed reasoning
    {
      "id": "gpt-4o-mini",
      "owned_by": "openai",
      "baseUrl": "https://api.openai.com/v1",
      "apiMode": "openai-responses",
      "reasoning_effort": "high",
      "extra": {
        "reasoning": {
          "summary": "detailed"
        }
      }
    }
    
    // Gemini with thought summaries
    {
      "id": "gemini-3-flash-preview",
      "owned_by": "gemini",
      "baseUrl": "https://generativelanguage.googleapis.com",
      "apiMode": "gemini",
      "extra": {
        "generationConfig": {
          "thinkingConfig": {
            "includeThoughts": true
          }
        }
      }
    }
  5. Open the Visual Configuration UI

    main

    The extension provides a visual interface to manage global settings, providers, and models without manual JSON editing. You can open it via:

    1. Command Palette: Press Ctrl+Shift+P (macOS: Cmd+Shift+P), search for OAICopilot: Open Configuration UI, and select it.
    2. Status Bar: Click the OAICopilot item in the VS Code status bar (bottom right).

    Note: If you use the Configuration UI, the global oaicopilot.baseUrl and API keys configured in JSON settings will be ignored in favor of the UI-managed settings.

  6. Configure Thinking/Reasoning modules for specific providers

    main

    To display the Thinking module in Copilot, you must use provider-specific parameters via the extra field or specific reasoning fields, depending on the apiMode and provider.

    OpenAI Responses

    Set apiMode: "openai-responses" and use the extra field for reasoning summary configuration:

    {
      "id": "gpt-4o-mini",
      "owned_by": "openai",
      "baseUrl": "https://api.openai.com/v1",
      "apiMode": "openai-responses",
      "reasoning_effort": "high",
      "extra": {
        "reasoning": {
          "summary": "detailed"
        }
      }
    }

    Gemini

    Set apiMode: "gemini" and use the extra field to enable thinkingConfig within generationConfig:

    {
      "id": "gemini-3-flash-preview",
      "owned_by": "gemini",
      "baseUrl": "https://generativelanguage.googleapis.com",
      "apiMode": "gemini",
      "extra": {
        "generationConfig": {
          "thinkingConfig": {
            "includeThoughts": true
          }
        }
      }
    }
    // Gemini Thinking Example
    {
      "id": "gemini-3-flash-preview",
      "owned_by": "gemini",
      "baseUrl": "https://generativelanguage.googleapis.com",
      "apiMode": "gemini",
      "extra": {
        "generationConfig": {
          "thinkingConfig": {
            "includeThoughts": true
          }
        }
      }
    }
  7. Quick Start: Install and Configure OAI Compatible Provider

    main

    To use any OpenAI, Ollama, Anthropic, or Gemini compatible API in VS Code's GitHub Copilot Chat, follow these steps:

    1. Install: Install the OAI Compatible Provider for Copilot extension from the VS Code Marketplace.
    2. Configure Base URL and Models: Open VS Code settings and configure oaicopilot.baseUrl and oaicopilot.models.
    3. Access Model Selector: Open the GitHub Copilot Chat interface, click the model selector, and choose "Manage Models...".
    4. Select Provider: Choose the "OAI Compatible" provider.
    5. Set API Key: Enter your API key (it will be saved locally).
    6. Add Models: Select the models you want to appear in the model selector.
    "oaicopilot.baseUrl": "https://api-inference.modelscope.cn/v1",
    "oaicopilot.models": [
        {
            "id": "Qwen/Qwen3-Coder-480B-A35B-Instruct",
            "owned_by": "modelscope",
            "context_length": 256000,
            "max_tokens": 8192,
            "temperature": 0,
            "top_p": 1
        }
    ]
  8. Use the Visual Configuration UI

    main

    The extension provides a visual interface to manage providers and models without manual JSON editing.

    Opening the UI

    • Command Palette: Press Ctrl+Shift+P (or Cmd+Shift+P) and search for OAICopilot: Open Configuration UI.
    • Status Bar: Click the OAICopilot item in the bottom-right corner of VS Code.

    Workflow

    1. Add a Provider: In Provider Management, click "Add Provider", enter a Provider ID (e.g., modelscope), Base URL, API Key, and API Mode, then click "Save".
    2. Add a Model: In Model Management, click "Add Model", select your Provider, enter the Model ID, configure parameters (context length, max tokens, etc.), and click "Save Model".
    3. Activate Models: Open GitHub Copilot Chat, click the model picker, select "Manage Models...", choose "OAI Compatible", and select your configured models.
  9. Configure temperature for different use cases

    main

    The temperature parameter (range [0, 2]) controls output randomness.

    • 0.0 - 0.3 (Low): Focused and deterministic. Best for precise code generation and debugging.
    • 0.4 - 0.7 (Moderate): Balanced. Good for architecture design and brainstorming.
    • 0.7 - 2.0 (High): Creative and varied. Best for open-ended questions.

    Best Practices:

    • Set to 0 for consistent, deterministic code suggestions (aligns with GitHub Copilot default).
    • Set to 1.0 for thinking-enabled models to ensure optimal performance of the reasoning mechanism.
  10. Understand Anthropic content block types

    main

    Anthropic messages use a polymorphic AnthropicContentBlock type. A content block can be one of the following:

    • AnthropicTextBlock: Standard text content.
    • AnthropicImageBlock: Image content provided via base64 encoding.
    • AnthropicThinkingBlock: Represents the model's internal reasoning/thinking process.
    • AnthropicToolUseBlock: Represents a tool call initiated by the assistant.
    • AnthropicToolResultBlock: Represents the result of a tool execution, provided back to the assistant.
  11. Configure Anthropic prompt caching

    main

    When using apiMode: "anthropic", you can control prompt caching via the cache_control boolean property in the model configuration.

    When cache_control is true (the default), the provider:

    1. Converts the system message into a structured array and marks it with cache_control: { type: "ephemeral" }.
    2. Marks the last entry of tools with cache_control: { type: "ephemeral" }.
    3. Honors in-message cache_control markers emitted by the host (Copilot) by converting LanguageModelDataPart with mimeType === "cache_control" into a real Anthropic cache_control field.

    Set this to false if your upstream provider rejects cache_control markers.

  12. Supported API Modes for Language Models

    main

    The provider supports several distinct API modes, which determine how chat messages are converted and sent to the backend:

    • openai (Default): Uses the standard OpenAI /chat/completions endpoint. It includes stream_options: { include_usage: true } in the request.
    • ollama: Uses the Ollama native /api/chat endpoint. Messages are converted to the Ollama format.
    • anthropic: Uses the Anthropic /messages endpoint. It handles message conversion and optional cache control.
    • gemini: Uses the Google Gemini generateContent endpoint. It converts messages into Gemini's contents and systemInstruction format.
    • openai-responses: Uses the OpenAI Responses API. This mode supports stateful conversations by using a previous_response_id to avoid sending the full chat history in every request. It uses a special stateful marker (application/vnd.oaicopilot.stateful-marker) to track conversation state.