AI Cookbook

The OpenAI Responses API is a superset of the Chat Completions API, offering all existing Chat Completions functionality plus additional advanced features. It provides a simplified interface for various interaction types and improved management of conversation states.

Key Features

• Simplified Interface: Streamlines different interaction types.
• Native Tool Support: Built-in capabilities for Web search, File search, and Computer use.
• New Roles: Includes a new developer role.
• Advanced Capabilities: Improved support for reasoning models and built-in vector search functionality.
• Efficiency: Features that previously required multiple API calls can often be handled in a single call.

This tutorial demonstrates how to build an AI agent that enhances its capabilities by combining multiple knowledge sources. The pattern involves an agent that intelligently decides between three primary retrieval methods to provide comprehensive, well-cited answers:

1. Internal Knowledge Base: Accessing curated content like handbooks, policies, or RAG pipelines.
2. Single Page Retrieval: Fetching and extracting content from specific URLs provided by the user or identified by the agent.
3. Web Search: Using broader internet search capabilities to find current or external information not present in internal documentation.

The Model Context Protocol (MCP) is a standardized protocol designed to allow Large Language Models (LLMs) to interact with external tools and services. Rather than implementing custom function-calling logic for every new tool, MCP provides a universal standard for these interactions.

This course focuses on the technical implementation for Python developers, specifically:

• Understanding MCP technical architecture.
• Building custom MCP servers using the Python SDK.
• Integrating MCP servers into production Python applications and agent systems.

The software is built using the following stack:

• Backend: Python 3.12+
• Frontend: TypeScript
• Web Frameworks: FastAPI (services), Next.js 15 (dashboard)
• Databases: PostgreSQL 16 (Neon for non-prod, RDS for prod)
• Queues: Redis 7 with RQ (for jobs < 1 minute), Temporal (for longer jobs)
• Object Storage: S3 with KMS-managed keys
• Vector Storage: LanceDB
• Observability: OpenTelemetry, Grafana Tempo, Loki, and Grafana Cloud
• CI/CD: GitHub Actions and ArgoCD (deployed to k8s)

NDCG@10 is a retrieval metric used to evaluate how well a retriever ranks relevant documents. Unlike Precision or Recall, NDCG rewards placing highly relevant documents at the top of the results list and penalizes placing them lower.

Key characteristics:

• Position Matters: A relevant document at rank 1 contributes more to the score than the same document at rank 10.
• Normalized: The score is normalized against an 'Ideal DCG' (IDCG), meaning a perfect ranking always results in a score of 1.0.
• Range: The metric is bounded between $[0, 1]$.
• Standardization: NDCG@10 is the industry standard for retrieval leaderboards (e.g., BEIR, MS MARCO) because users typically focus on the first page of results.

The Docling processing pipeline follows these stages:

1. Document parsing (format-specific backend)
2. Layout analysis (AI-powered)
3. Table structure recognition
4. Metadata extraction
5. Content organization/structuring
6. Export formatting (HTML, Markdown, JSON, or plain text)

Core AI Models

• Layout Analysis: Based on RT-DETR (Real-Time Detection Transformer) architecture. Trained on DocLayNet, it processes pages at 72 dpi in under a second on standard CPUs.
• Table Structure Recognition: Uses TableFormer to handle complex layouts (spanning cells, hierarchical headers, etc.). Processes tables in 2-6 seconds on CPU.
• OCR: Integrates EasyOCR for text extraction from images (operates at 216 dpi; ~30 seconds per page).

When building a retrieval system, choose between dense retrieval and BM25 based on the query type:

Recommendation: Use a hybrid approach (e.g., Reciprocal Rank Fusion/RRF) to combine the strengths of both.

The Model Context Protocol (MCP) uses a client-host-server architecture to separate concerns and allow modular, composable systems.

• MCP Hosts: Applications (e.g., Claude Desktop, IDEs, or custom Python apps) that consume data via MCP.
• MCP Clients: Protocol clients that maintain 1:1 connections with servers.
• MCP Servers: Lightweight programs that expose specific capabilities (tools, resources, prompts).
• Local Data Sources: Files, databases, or services on your machine accessed by servers.
• Remote Services: External systems accessed by servers via APIs over the internet.

The data pipeline consists of a 'hot path' for real-time dashboards and a 'cold path' for billing and trend reports. The architecture flows as follows:

• Hot Path: Customer events (Kafka) $\rightarrow$ Stream processor (Flink) $\rightarrow$ hot dashboards (sub-minute latency).
• Cold Path: Customer events (Kafka) $\rightarrow$ Hourly micro-batch (Airflow) $\rightarrow$ warehouse (BigQuery) $\rightarrow$ Nightly batch ETL (Airflow) $\rightarrow$ aggregates table (Postgres).

The billing service is a FastAPI application located at services/billing. It utilizes the following stack:

• PostgreSQL: Source of truth for subscriptions, invoices, and payment methods.
• Stripe: Payment processing.
• Redis: Used for idempotency keys (24-hour TTL) and webhook deduplication.
• RQ Worker: Processes Stripe webhooks asynchronously.

When using MCP with OpenAI, the tool execution follows a specific lifecycle:

1. Tool Registration: The MCP client converts MCP-defined tools into OpenAI's specific function calling format.
2. Tool Choice: OpenAI determines which tool is required based on the user's query.
3. Tool Execution: The MCP client executes the selected tool on the MCP server and captures the results.
4. Context Integration: The tool results are fed back into the OpenAI context to generate the final response.