DOCSEARCHbyAlgolia
Submit Docs

GPT4Free (g4f)

repository·main·Indexed 41 days ago

https://github.com/gpt4free/g4f.dev

A collection of free AI endpoints for text and image generation. It provides a Python client with synchronous and AsyncClient interfaces compatible with the OpenAI API, a JavaScript client for browser-based use without a backend, and a CLI. Includes the cake-baker CLI, a Go implementation of a proof-of-work protocol to earn credits via brute-force hashing.

Tokens
63.6K
Snippets
214
Records
282
Agent score
52%

What's inside GPT4Free

  1. Overview of the API Proxy Server

    main

    The API Proxy Server is a Cloudflare Worker script that acts as a gateway between clients and various AI providers. It provides a unified interface for accessing multiple AI models while offering built-in features such as:

    • Caching: Reduces latency and provider load.
    • Rate Limiting: Protects against abuse.
    • Security: Includes content-length validation, CORS management, and API key rotation.
    • Routing: Forwards requests to specific providers or handles simplified query interfaces.

  2. Manage conversation history in G4F CLI

    main

    The CLI maintains context between queries by default. You can interact with the history or reset it using the following methods:

    • Continue a conversation: Simply run a new prompt; the CLI uses the existing context.
      g4f client "Now explain it like I'm five"
    - **Clear history**: Use the `-C` or `--clear-history` flag to start a fresh session without previous context.
  ```bash
g4f client -C "New conversation"
    • Custom conversation file: Specify a custom path for the conversation state using --conversation-file FILE.

    By default, the conversation state is saved to ~/.config/g4f/conversation.json.

  3. Proxy Server Security and Rate Limiting

    main

    The proxy implements several security layers to ensure stability and prevent abuse:

    • Rate Limiting: API paths are limited to 1 request per 10 seconds.
    • Content-Length Validation: Requests are checked against size limits to prevent large payload attacks. The default MAX_CONTENT_LENGTH is 50KB, while certain endpoints allow an extended CUSTOM_MAX_LENGTH of 100KB.
    • CORS Handling: Manages Cross-Origin Resource Sharing via CORS_HEADERS to allow web-based clients to interact with the proxy.
    • API Key Rotation: The server automatically selects from multiple API keys to distribute load and maintain access.
    • Content-Type Validation: Ensures incoming requests contain expected content types.

  5. Use the Chat Completions API

    main

    All chat-style interactions are handled via the {base_url}/chat/completions endpoint.

    Authentication

    • For providers marked as "none required", you do not need to pass an API key. You can pass a dummy string like 'secret' if your client requires a value.
    • For providers requiring a key (e.g., https://g4f.space/v1), supply the api_key in your client configuration or request headers.

    Payload Schema

    Requests should follow a standard chat completion format:

    • model (string): The identifier for the model (e.g., gpt-4o, gpt-oss-120b).
    • messages (array): An array of message objects with role (system, user, or assistant) and content (string).
    • temperature (number, optional): Controls randomness.
    {
  "model": "gpt-4o",
  "temperature": 0.9,
  "messages": [{"role": "user", "content": "Hello, how are you?"}]
}

  6. Implement conversation memory manually

    main

    To maintain a coherent conversation without using the built-in conversation object, you must manually manage a history list. Append both user messages (role: "user") and assistant responses (role: "assistant") to this list and pass the entire list to the messages parameter in every subsequent call.

    import asyncio
from g4f.client import AsyncClient

class Conversation:
    def __init__(self):
        self.client = AsyncClient()
        self.history = [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            }
        ]
    
    def add_message(self, role, content):
        self.history.append({"role": role, "content": content})
    
    async def get_response(self, user_message):
        self.add_message("user", user_message)
        
        response = await self.client.chat.completions.create(
            model="gpt-4o-mini",
            messages=self.history,
            web_search=False
        )
        
        assistant_response = response.choices[0].message.content
        self.add_message("assistant", assistant_response)
        
        return assistant_response

async def main():
    conversation = Conversation()
    # ... usage loop ...

asyncio.run(main())

  7. Implement an MCP server using JSON-RPC 2.0

    main

    G4F integrates with MCP servers via JSON-RPC 2.0 over HTTP POST. To be compatible, your server must respond to POST requests at its base URL with the following structure:

    Request Requirements

    All requests must include:

    • jsonrpc: Must be exactly "2.0"
    • method: The name of the method (e.g., "tools/list" or "tools/call")
    • params: An object containing method parameters
    • id: A unique identifier (number or string)

    Security Requirements

    • CORS: Your server must allow CORS requests from the G4F chat UI.
    • Validation: The server is responsible for validating all tool inputs.

  8. How conversation context is handled by different providers

    main

    G4F providers handle conversation context in two primary ways. Understanding which method a provider uses is critical for maintaining stateful chats:

    1. Message History Providers (Most Common): These providers require you to manually maintain and pass the entire conversation history in the messages parameter. You must append every user message and every assistant response to a local list and send that full list with every new request.

    2. Conversation Object Providers (e.g., OpenaiAccount): These providers manage state on their end. Instead of sending the full history, you pass the current message and a conversation object (retrieved from the previous response). This is more efficient for long conversations as you only send the latest message and a reference ID.

  9. Use ToolSupportProvider for structured output in PydanticAI

    main

    If a model or provider does not natively support tool calls or structured outputs (like Pydantic models), use ToolSupportProvider from g4f.providers.tool_support. This acts as a bridge that ensures the agent can format raw or unstructured model responses into the expected structured format (e.g., a Pydantic BaseModel).

    from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models import ModelSettings
from g4f.integration.pydantic_ai import AIModel
from g4f.providers.tool_support import ToolSupportProvider

class MyModel(BaseModel):
    city: str
    country: str

agent = Agent(
    AIModel(
        "OpenaiChat:gpt-4o",
        ToolSupportProvider
    ),
    result_type=MyModel,
    model_settings=ModelSettings(temperature=0)
)

result = agent.run_sync('European city with the bear.')
print(result.data)
    from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models import ModelSettings
from g4f.integration.pydantic_ai import AIModel
from g4f.providers.tool_support import ToolSupportProvider

# Define a custom model for structured output
class MyModel(BaseModel):
    city: str
    country: str

# Create the agent using ToolSupportProvider to bridge the gap
agent = Agent(AIModel(
    "OpenaiChat:gpt-4o",
    ToolSupportProvider
), result_type=MyModel, model_settings=ModelSettings(temperature=0))

if __name__ == '__main__':
    result = agent.run_sync('European city with the bear.')
    print(result.data)  # Structured output of city and country
    print(result.usage()) # Usage statistics

  10. How adaptive throttling works in cake-baker

    main

    To manage load, the baker implements an adaptive throttle that scales the sleep interval between batches based on daily progress toward the limit:

    • Below 50% of daily limit: Sleeps for the duration specified by -interval ms.
    • Between 50% and 90%: The interval scales linearly up to 60 seconds.
    • At or above 90%: Sleeps for 60 seconds.
    • Daily limit reached: The baker sleeps for 10 minutes before retrying.

  11. Proxy Caching Strategy

    main

    The proxy uses different caching durations based on the type of content being served:

    • Static contents: Uses CACHE_FOREVER (1 year).
    • API responses: Uses a time-limited policy of 4 hours (CACHE_CONTROL).
    • Media/Files: Uses a cache-based distribution model.

  12. Compare G4F File API and MarkItDown

    main

    Choose the appropriate API based on your workflow requirements:

    FeatureG4F File APIMarkItDown API
    Upload Files✅ Yes✅ Yes
    Web URL Downloads✅ Yes via downloads.json❌ No
    SSE Progress Streaming✅ Yes❌ No
    Markdown/Text OutputRaw/structuredClean markdown/plain text
    Bucket/File Management✅ Multi-file❌ Single-file only
    Best Use CaseMulti-step pipelines, large workflowsQuick extraction, inline usage