DOCSEARCHbyAlgolia
Submit Docs

deepreasoning

repository·main·Indexed 26 days ago

https://github.com/winfunc/deepreasoning

A high-performance LLM inference API and Chat UI (version 0.1.0) that integrates DeepSeek R1's Chain of Thought (CoT) reasoning traces with Anthropic Claude models into a single unified stream. Built with Rust, it provides a REST API supporting streaming and non-streaming responses, token usage tracking, and cost statistics via a configurable config.toml file.

Tokens
6.3K
Snippets
18
Records
30
Agent score
90%

What's inside deepreasoning

  3. Install and build DeepReasoning

    main

    To install DeepReasoning, clone the repository and build the project using Cargo.

    Prerequisites:

    • Rust 1.75 or higher
    • DeepSeek API key
    • Anthropic API key
    git clone https://github.com/getasterisk/deepreasoning.git
cd deepreasoning
cargo build --release

  4. Configure DeepReasoning via config.toml

    main

    Create a config.toml file in the project root to configure server settings and pricing for usage tracking.

    [server]
host = "127.0.0.1"
port = 3000

[pricing]
# Configure pricing settings for usage tracking

  5. Deploy the deepreasoning API using Docker Compose

    main

    You can deploy the deepreasoning_api service using Docker Compose. The service builds from the local directory and exposes the API on 127.0.0.1:1337.

    To configure the API, you must provide a local config.toml file, which is mounted into the container at /usr/local/bin/config.toml.

    services:
  api:
    build: .
    container_name: deepreasoning_api
    restart: unless-stopped
    ports:
      - "127.0.0.1:1337:1337"
    volumes:
      - ./config.toml:/usr/local/bin/config.toml
    networks:
      - deepreasoning_network

networks:
  deepreasoning_network:
    name: deepreasoning_network

  6. Run the DeepReasoning service

    main

    The DeepReasoning service is a high-performance LLM inference API that integrates DeepSeek R1's Chain-of-Thought (CoT) reasoning traces with Anthropic Claude models. It provides a REST API for chat interactions, supporting both streaming and non-streaming responses, token usage tracking, and cost statistics.

    To run the service, ensure you have a config.toml file available in the working directory. If the configuration file is missing, the service will attempt to use default settings.

    API Endpoint

    • POST /: The primary endpoint for chat interactions, handled by handlers::handle_chat.

  7. Use the DeepReasoning API (Basic Example)

    main

    Send a POST request to the DeepReasoning server. You must provide your DeepSeek and Anthropic API keys in the request headers using X-DeepSeek-API-Token and X-Anthropic-API-Token.

    import requests

response = requests.post(
    "http://127.0.0.1:1337/",
    headers={
        "X-DeepSeek-API-Token": "<YOUR_DEEPSEEK_API_KEY>",
        "X-Anthropic-API-Token": "<YOUR_ANTHROPIC_API_KEY>"
    },
    json={
        "messages": [
            {"role": "user", "content": "How many 'r's in the word 'strawberry'?"}
        ]
    }
)

print(response.json())

  8. Stream responses from the DeepReasoning API

    main

    To receive a stream of responses (combining R1's CoT and Claude's response), set "stream": true in your JSON request body. The server emits data in the data: <json_string> format.

    import asyncio
import json
import httpx

async def stream_response():
    async with httpx.AsyncClient() as client:
        async with client.stream(
            "POST",
            "http://127.0.0.1:1337/",
            headers={
                "X-DeepSeek-API-Token": "<YOUR_DEEPSEEK_API_KEY>",
                "X-Anthropic-API-Token": "<YOUR_ANTHROPIC_API_KEY>"
            },
            json={
                "stream": True,
                "messages": [
                    {"role": "user", "content": "How many 'r's in the word 'strawberry'?"}
                ]
            }
        ) as response:
            response.raise_for_status()
            async for line in response.aiter_lines():
                if line:
                    if line.startswith('data: '):
                        data = line[6:]
                        try:
                            parsed_data = json.loads(data)
                            if 'content' in parsed_data:
                                content = parsed_data.get('content', '')[0]['text']
                                print(content, end='',flush=True)
                            else:
                                print(data, flush=True)
                        except json.JSONDecodeError:
                            pass

if __name__ == "__main__":
    asyncio.run(stream_response())

  9. Reference: API Request Body Options

    main

    The DeepReasoning API accepts the following configuration options in the request body:

    {
    "stream": false,
    "verbose": false,
    "system": "Optional system prompt",
    "messages": [...],
    "deepseek_config": {
        "headers": {},
        "body": {}
    },
    "anthropic_config": {
        "headers": {},
        "body": {}
    }
}

  10. Initialize the DeepSeekClient

    main

    To interact with DeepSeek's AI models, initialize a DeepSeekClient by providing your API token. The client handles authentication, request construction, and response parsing for both streaming and non-streaming modes.

    use deepreasoning::clients::DeepSeekClient;

let client = DeepSeekClient::new("your-api-key".to_string());

  11. Perform a non-streaming chat request with DeepSeekClient::chat

    main

    Use the chat method to send a single request and receive a complete DeepSeekResponse. This method is asynchronous and requires a vector of Message objects and an ApiConfig.

    Parameters:

    • messages: Vec<Message> containing the conversation history.
    • config: &ApiConfig containing request parameters like model, max_tokens, and temperature via the body field.

    Returns:

    • Result<DeepSeekResponse>: The full response containing choices, usage statistics, and model metadata.
    use crate::clients::DeepSeekClient;
use crate::models::{ApiConfig, Message};

// Initialize the client
let client = DeepSeekClient::new("your-api-key".to_string());

// Prepare messages and configuration
let messages = vec![Message {
    role: "user".to_string(),
    content: "Hello, how are you?".to_string(),
}];

let config = ApiConfig::default();

// Make a non-streaming request
let response = client.chat(messages, &config).await?;
println!("Response: {:?}", response.choices[0].message.content);

  12. Perform a streaming chat request with DeepSeekClient::chat_stream

    main

    Use the chat_stream method to receive real-time updates from the model. This method returns a pinned, boxed stream of StreamResponse objects. Each chunk in the stream contains a delta which may include content or reasoning_content (for reasoning models).

    Parameters:

    • messages: Vec<Message> containing the conversation history.
    • config: &ApiConfig containing request parameters.

    Returns:

    • Pin<Box<dyn Stream<Item = Result<StreamResponse>> + Send>>>: A stream of response chunks.