DOCSEARCHbyAlgolia
Submit Docs

LLM4Free Python Toolkit

repository·main·Indexed 43 days ago

https://github.com/oevortex/llm4free

A unified Python toolkit providing an OpenAI-compatible interface to over 40 free and paid AI models, including LLMs, TTS, TTI, and STT. It features a unified Client for automatic provider selection and failover, a FastAPI-based API server, and a search stack supporting engines like DuckDuckGo, Bing, and Perplexity. Additionally, it includes auxiliary tools for temporary email and phone number generation, YouTube transcription, and a rich CLI for web search and utility tasks.

Tokens
117.5K
Snippets
400
Records
485
Agent score
55%

What's inside LLM4Free

  1. Overview of LLM4Free components and architecture

    main

    LLM4Free is a modular ecosystem designed to provide access to various LLM, TTS (Text-to-Speech), TTI (Text-to-Image), and STT (Speech-to-Text) providers. The project is organized into several core functional areas:

    • Unified Python Client (llm4free/client.py): A high-level client providing intelligent provider/model resolution and automatic failover for chat and image tasks.
    • Command Line Interface (llm4free/cli.py): A rich CLI (built on swiftcli) for interacting with search engines (DuckDuckGo, Bing, etc.), weather utilities, and other tools.
    • OpenAI-compatible API Server (llm4free/server/): A FastAPI-based server that proxies LLM4Free providers using the OpenAI API format.
    • Search Stack (llm4free/search/): Shared abstractions for engines including ddg (DuckDuckGo), bing, yahoo, brave, mojeek, wikipedia, and serpbase.
    • Provider Implementations (llm4free/llm/): Contains both no-auth and authenticated providers (under llm4free/llm/Auth/), covering AI search, TTI, TTS, and STT.
    • Auxiliary Toolkits (llm4free/Extra/): Includes GGUF conversion, weather clients, temp mail, and Git/YouTube utilities.
    • Scout (llm4free/scout): An HTML parser and crawler module.
    • Media Generation: Dedicated modules for TTI, TTS, and STT.

  2. Overview of the YTToolkit module structure

    main

    The YTToolkit module provides a suite of tools for interacting with YouTube data, including video downloading, transcript extraction, and metadata retrieval. The toolkit is organized into several functional modules:

    • Video Downloader (YTdownloader.py): Handles downloading YouTube videos with options for format and quality.
    • Transcript Retriever (transcriber.py): Uses the InnerTube API for reliable transcript extraction.
    • Data Extraction (ytapi):
      • channel.py: Channel metadata and interaction.
      • video.py: Video information extraction.
      • query.py: Advanced search capabilities.
      • extras.py: Trending and category-based video retrieval.
      • playlist.py: Playlist metadata extraction.
      • captions.py: Wrapper for caption/transcript extraction.
      • suggestions.py: Search autocomplete and trending searches.
      • shorts.py: YouTube Shorts functionality.
      • hashtag.py: Hashtag-related video discovery.

  3. Overview of LLM4Free OpenAI-Compatible Providers

    main
    LLM4Free provides a standardized interface to interact with multiple AI providers using the OpenAI API structure. This allows developers to use the same code patterns across different providers and switch between them with minimal code changes, leveraging existing OpenAI-compatible tools and libraries.

  4. Browse LLM4Free Awesome Prompts

    main

    The llm4free project includes a curated collection of specialized prompts categorized by persona and utility. These prompts can be used to transform an LLM into specific roles such as professional experts, creative personas, advisors, or experimental interaction models.

    Prompt Categories

    • Professional Roles: Includes personas like Position Interviewer, Recruiter, Financial Analyst, Doctor, and Product Manager.
    • Entertainment & Creative: Includes Character from Movie/Book/Anything, Text Based Adventure Game, Midjourney Prompt Generator, and ASCII Artist.
    • Advisors & Coaches: Includes Travel Guide, Motivational Coach, Relationship Coach, Chef, and Legal Advisor.
    • Alternative Models: Experimental interaction modes like Free-mode, AI Trying to Escape the Box, Gaslighter, and Unconstrained AI Model DAN.
    • Specialized Assistants: Niche helpers like Etymologist, Statistician, Prompt Generator, and StackOverflow Post simulator.

  5. Compare Search Engine Capabilities

    main

    LLM4Free supports various search categories across different engines. Use the following table to determine which engine meets your requirements:

    CategoryEngines
    textDuckDuckGo, Bing, Brave, Yahoo, Mojeek, Wikipedia, SerpBase
    imagesDuckDuckGo, Bing, Brave, Yahoo, SerpBase
    videosDuckDuckGo, Brave, Yahoo
    newsDuckDuckGo, Bing, Brave, Yahoo
    suggestionsDuckDuckGo, Bing, Brave, Yahoo
    weatherDuckDuckGo, Yahoo
    answersDuckDuckGo
    translateDuckDuckGo
    mapsDuckDuckGo

    Note: SerpBase requires an API key (required_auth = True). All other engines are available without authentication.

  6. Understand the OpenAICompatibleProvider interface

    main

    All providers in llm4free inherit from the OpenAICompatibleProvider base class. This ensures a standardized interface for interacting with different LLM backends. To use a provider, you primarily interact with the chat method, which accepts a list of messages and an optional stream flag.

    class OpenAICompatibleProvider:
    AVAILABLE_MODELS = []  # List of available models
    
    def chat(self, messages, stream=False):
        """Send chat messages and get response."""
        pass

  7. Project Directory Structure and Architecture

    main

    When extending the project or locating specific components, follow this directory structure:

    • Provider implementations: llm4free/Provider/
    • Search engines: llm4free/search/
    • Utilities: llm4free/Extra/
    • Tests: tests/providers/
    • Documentation: docs/

  8. Understand the LLM4Free Provider Architecture

    main

    LLM4Free uses a hierarchical provider system to translate user prompts into API calls. There are two main types of providers:

    1. Native Providers: Direct implementations from scratch, used for services with unique or non-standard APIs. These inherit from AIbase.Provider.
    2. OpenAI-Compatible Providers: For services that mimic the OpenAI API format. These inherit from OpenAICompatibleProvider.

    Key Concepts:

    • Provider: A class that handles the translation of prompts to API requests.
    • Model: The specific AI model identifier (e.g., gpt-4).
    • Session: An HTTP client (using curl_cffi) used for reliable API communication.
    • Conversation: An optional mechanism to preserve context across multiple turns.

  9. Resolve models using the Provider/Model format

    main

    The server uses a dynamic resolution strategy for models. To target a specific model, use the format ProviderName/model_name.

    Examples:

    • ChatGPT/gpt-4o
    • Cloudflare/@cf/meta/llama-4-scout-17b-16e-instruct

    Key Behaviors:

    • The server resolves available providers at startup from llm4free.llm, llm4free.TTI, and llm4free.TTS.
    • Providers requiring authentication are excluded by default.
    • You can verify available models by querying the /v1/models endpoint.

  10. Debug the unified Client with exclusion and tracing

    main

    When using the unified Client, you can improve debugging by:

    1. Excluding flaky providers: Use exclude=[...] in the constructor to skip specific providers.
    2. Tracing the provider: Pass print_provider_info=True to see a banner, or inspect client.chat.completions.last_provider after a call to see which provider served the request.
    from llm4free.client import Client

# Exclude a flaky provider
client = Client(exclude=["SomeFlakyProvider"])

# Trace which provider answered
client = Client(print_provider_info=True)
resp = client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "Hi"}])
print("Served by:", client.chat.completions.last_provider)

  11. Enable command chaining and pipelines

    main

    SwiftCLI supports chaining commands where the output of one command is passed to the next.

    1. Enable chaining globally with app.enable_chaining(True).
    2. For complex workflows, use @app.group(chain=True) to create a command pipeline where commands are executed in sequence.
    # Global chaining
app.enable_chaining(True)

@app.command()
def step1():
    return {"data": "from step1"}

@app.command()
def step2(data: dict):
    return {"result": f"Processed {data['data']}"}

# Command Pipelines
@app.group(chain=True)
def process():
    """Data processing pipeline"""
    pass

@process.command()
def extract():
    return {"data": [1, 2, 3]}

@process.command()
def transform(data: dict):
    return {"data": [x * 2 for x in data["data"]]}

@process.command()
def load(data: dict):
    print(f"Loading: {data}")

# Usage: python app.py process extract transform load

  12. Use Text-to-Image (TTI) providers in LLM4Free

    main

    LLM4Free provides a unified interface for generating images from text prompts using various third-party providers. All TTI providers follow an OpenAI-inspired API, exposing an images.create() method that returns an ImageResponse object.

    Key features include:

    • Unified API: Use the same method signature across different providers.
    • Flexible Output: Request images as URLs or base64 encoded bytes via the response_format parameter.
    • Model Discovery: Query available models using provider.models.list().
    • Automatic Hosting: Generated images are automatically uploaded to a fallback chain of hosting services (catbox.moe, 0x0.st, etc.) when requesting URLs.
    from llm4free.Provider.TTI import PollinationsAI

client = PollinationsAI()
response = client.images.create(
    model="flux",
    prompt="A futuristic city skyline at sunset",
    n=2,
    response_format="url"
)