LlamaDeploy Documentation

The project is organized into deployment definitions, workflows, and a frontend:

• python_fullstack.yaml: The deployment definition. It instructs LlamaDeploy on which services to deploy and how to retrieve their source code.
• workflows/: Contains the core logic:
  • rag_workflow.py: Implements RAG using Qdrant vector store, retrieval, RankGPT reranking, and response synthesis.
  • agent_workflow.py: An advanced workflow that incorporates the RAG system and adds agentic capabilities.
• frontend/: A Reflex application:
  • frontend/frontend/frontend.py: Defines the chat UI.
  • frontend/frontend/state.py: Manages state and connects to the LlamaDeploy API server to communicate with workflows.
  • frontend/frontend/style.py: Manages UI styling.

LlamaDeploy is composed of several services that enable multi-agent applications to run and communicate. The architecture relies on the following interaction model:

• Service: A wrapper around a workflow that endlessly processes incoming requests.
• Task: The unit of work representing a request for an operation and its corresponding response.
• Message Queue: The communication layer where services pull and publish messages.
• Control Plane: An internal component that manages ongoing tasks, maintains internal state, tracks available services, and routes Tasks to the appropriate Service.
• Deployment: A collection of these components configured together.
• API Server: The entry point that manages multiple deployments and exposes an HTTP API for administration and querying.

A bootstrapped LlamaDeploy project typically contains the following structure:

• deployment.yml: The declarative configuration file for your deployment. Use this to set environment variables (like env.OPENAI_API_KEY), manage services, and define Python dependencies.
• src/workflow.py: The Python file containing your Workflow logic (e.g., using LlamaIndex Workflows and @step decorators).
• ui/: (Optional) A Next.js/React frontend scaffold designed to interact with your deployment.

The Client provides access to core components to manage the deployed system. The primary component is:

• apiserver: Used to interact with the API server.

Requirement: To use apiserver functionalities, the API Server must be running and its URL (default: http://localhost:4501) must be reachable by the client.

Follow these steps to deploy and query your workflow:

1. Start the API server (ensure OPENAI_API_KEY and LLAMA_CLOUD_API_KEY are set):

   python -m llama_deploy.apiserver

2. Deploy the configuration using llamactl:

   llamactl deploy deployment.yml

3. Run a query against the deployment:

   llamactl run --deployment LlamaCloud_LlamaDeploy_GoogleDrive --arg query '<YOUR QUERY>'

# Start server
python -m llama_deploy.apiserver

# Deploy
llamactl deploy deployment.yml

# Query
llamactl run --deployment LlamaCloud_LlamaDeploy_GoogleDrive --arg query '<YOUR QUERY>'

Before deploying, you must have the LlamaDeploy API server running. You can run it locally using the following command:

python -m llama_deploy.apiserver

Alternatively, you can use Docker to run the API server:

docker run -p 4501:4501 -v .:/opt/quickstart -w /opt/quickstart llamaindex/llama-deploy:main

To run a project locally, follow these three steps:

1. Start the API server: Run the server using Python:

   python -m llama_deploy.apiserver

   Or via Docker:

   docker run -p 4501:4501 -v "$PWD":/opt/app -w /opt/app llamaindex/llama-deploy:main

2. Deploy the workflow: In a separate terminal, use llamactl to deploy the configuration defined in your YAML file:

   llamactl deploy deployment.yml

3. Execute the workflow: Test your deployment by running a command with specific arguments:

   llamactl run --deployment <deployment_name> --arg <key>='<value>'

You can extend your deployment using the following patterns:

• Modify Logic: Edit src/workflow.py to change LLMs, add tools, or implement complex multi-step logic using LlamaIndex Workflows.
• Add Services: To add more workflows, duplicate a service block in deployment.yml and point it to your new workflow file.
• Manage Secrets: Use the env or env_files keys within each service definition in deployment.yml to set environment variables and secrets.
• Production Deployment: For production, containerize the deployment (e.g., using Google Cloud Run) to run and scale on Docker/K8s environments.