DOCSEARCHbyAlgolia
Submit Docs

AI DIAL Python SDK

repository·development·Indexed 42 days ago

https://github.com/epam/ai-dial-sdk

A framework for creating AI applications and model adapters compatible with the AI DIAL API, designed based on the Azure OpenAI API. Version 0.37.0 provides the DIALApp class (extending FastAPI) for routing requests, support for ChatCompletion and Embeddings implementations, and integrated OpenTelemetry telemetry. It includes examples for text-to-text, image-to-text, Langchain RAG, and text-to-image applications.

Tokens
7.1K
Snippets
23
Records
39
Agent score
52%

What's inside AI DIAL SDK

  1. Overview of the render-text application

    development

    The render-text application is a simple text-to-image DIAL application. It extracts text from the last user message's attachments and returns an image containing that rasterized text.

    The application provides the result in two ways:

    1. As an image attachment in the response message.
    2. As a Markdown image within the response text.

    When running via Docker, the application exposes the openai/deployments/render-text/chat/completions endpoint on port 5000.

  2. Resolve deployment IDs in requests

    development

    The request.deployment_id is resolved using the following priority order:

    1. The X-DIAL-OVERRIDE-NAME header.
    2. The deployment name from the path (for /openai/deployments/{deployment_name} requests).
    3. The X-DIAL-DEPLOYMENT-ID header (set by DIAL Core for /openai/v1 endpoints).

    Note: An /openai/v1 request with none of these headers set will result in a 500 Internal Server Error.

  3. How to create an Echo application with ChatCompletion

    development

    To create an application or model adapter, inherit from the ChatCompletion abstract class and implement the chat_completion method. This method receives a Request and a Response object.

    In the example below, we extract the last message from the request history and use response.create_single_choice() to generate a response containing that same text.

    import uvicorn
from aidial_sdk import DIALApp
from aidial_sdk.chat_completion import ChatCompletion, Request, Response

class EchoApplication(ChatCompletion):
    async def chat_completion(self, request: Request, response: Response) -> None:
        # Get last message (the newest) from the history
        last_user_message = request.messages[-1]

        # Generate response with a single choice
        with response.create_single_choice() as choice:
            # Fill the content of the response with the last user's content
            choice.append_content(last_user_message.text())

# DIALApp extends FastAPI to provide a user-friendly interface for routing requests
app = DIALApp()
app.add_chat_completion("echo", EchoApplication())

if __name__ == "__main__":
    uvicorn.run(app, port=5000)

  4. Choose a logging mode for the DIAL SDK

    development

    The SDK logs to stderr using Python's logging module. You must choose exactly one of three mutually exclusive logging modes based on your requirements:

    1. DIAL SDK Formatter (Default): Best if you want custom-shaped JSON or human-readable text logs without needing OpenTelemetry. Works with or without telemetry.
    2. OTel Log Correlation: Best if you want text logs that automatically include OpenTelemetry trace and span IDs. Requires aidial-sdk[telemetry] and OTEL_TRACES_EXPORTER=otlp.
    3. OTel Log Export: Best if you want OTel-native structured logs (fixed shape) sent to a collector or console. Requires aidial-sdk[telemetry] and setting OTEL_LOGS_EXPORTER to console or otlp.

  5. Set up the Langchain RAG example application

    development

    To run the Langchain RAG example locally, create a virtual environment, install the required dependencies, and run the application module. This example uses Chroma vector database and a RetrievalQA chain to process documents provided via attachments in the chat history.

    Local Setup Steps

    1. Create a virtual environment using Python 3.11.
    2. Activate the environment.
    3. Install dependencies from requirements.txt.
    4. Run the application using the -m app flag.

    Note: The application exposes the openai/deployments/simple-rag/chat/completions endpoint at port 5000 upon startup.

    python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m app

  6. Format user messages for the render-text application

    development

    To control how the application processes the input, the user message must include an attachment following this specific format:

    (base64|url),<text to render>

    This tells the application whether to expect a Base64 string or a URL, followed by the actual text that should be rasterized into the image.

  7. Enable OTel log correlation for text logs

    development

    To add OpenTelemetry trace context to your text logs using OTel's built-in format, you must enable tracing and correlation. This mode takes precedence over the DIAL SDK text formatter.

    Requirements:

    1. Install the telemetry extra: pip install aidial-sdk[telemetry]
    2. Initialize DIALApp with telemetry_config=TelemetryConfig().
    3. Set OTEL_TRACES_EXPORTER=otlp (correlation is a no-op without active tracing).
    4. Set OTEL_PYTHON_LOG_CORRELATION=true.

    You can customize the OTel text template using OTEL_PYTHON_LOG_FORMAT.

    OTEL_TRACES_EXPORTER=otlp
OTEL_PYTHON_LOG_CORRELATION=true
OTEL_PYTHON_LOG_FORMAT='%(asctime)s %(levelname)s trace_id=%(otelTraceID)s - %(message)s'

  8. Export logs using OTel-native structured logs

    development

    To use OTel's structured log pipeline, set the OTEL_LOGS_EXPORTER environment variable. This mode ignores all DIAL_SDK_LOG_FORMAT variables.

    Requirements:

    • aidial-sdk[telemetry] installed.
    • DIALApp(telemetry_config=TelemetryConfig()) initialized.

    Exporter Options:

    • otlp: Ships structured log records (with trace context) to an OTLP collector.
    • console: Emits each record as a single compact JSON object per line to stderr via the OTel console exporter.

    You can combine them: OTEL_LOGS_EXPORTER=otlp,console.

  9. Integrate the Echo application with DIAL Core

    development

    To use the Echo application within a DIAL ecosystem, you must integrate it into DIAL Core and invoke it via the DIAL API. Detailed implementation patterns for calling text-to-text applications can be found in the DIAL Cookbook.

    https://github.com/epam/ai-dial/blob/main/dial-cookbook/examples/how_to_call_text_to_text_applications.ipynb

  12. Configure custom JSON or Text logs with DIAL SDK Formatter

    development

    Use the DIAL SDK formatter to define your own log shapes. This mode is controlled by DIAL_SDK_LOG_FORMAT.

    Configuration Variables

    VariableDefaultPurpose
    DIAL_SDK_LOGWARNINGLog level for the aidial_sdk logger
    DIAL_SDK_LOG_FORMATtexttext (colored) or json (one line per record)
    DIAL_SDK_TEXT_LOG_FORMAT(see below)%-style format string for text mode
    DIAL_SDK_JSON_LOG_FORMAT(see below)JSON template for JSON mode

    Text Mode

    Default text rendering uses uvicorn's DefaultFormatter. Use %(levelprefix)s for colored, aligned levels.

    JSON Mode

    Provide a JSON document where every string leaf is a %-style format string. The SDK interpolates these and then performs json.dumps().

    Adding Trace/Span IDs in JSON Mode

    If tracing is enabled, JSON mode automatically adds otelTraceID, otelSpanID, otelTraceSampled, and otelServiceName fields. To rename them, reference them in your template (this suppresses the auto-added copy).

    # Custom Text Format
DIAL_SDK_TEXT_LOG_FORMAT='%(levelprefix)s %(name)s: %(message)s'

# Custom JSON Format with renamed trace_id
DIAL_SDK_JSON_LOG_FORMAT='{"level":"%(levelname)s","message":"%(message)s","trace_id":"%(otelTraceID)s"}'