Agno (formerly Phidata)

The LLM OS is a conceptual architecture where the LLM serves as the kernel process. Instead of just generating text, the LLM coordinates various resources to solve problems:

• Memory: Provided via vector databases like PgVector for long-term knowledge.
• Computation Tools: The LLM can use existing software infrastructure such as calculators, Python interpreters, and shell tools.
• Internet/Research: The LLM can browse the web and perform research (e.g., using Exa).
• Communication: The LLM can communicate with other LLMs.

In this specific implementation, the LLM can read/generate text, browse the internet, use software tools, and communicate with other LLMs.

When building Assistants with Phidata, it is important to distinguish between these two components:

• Knowledge Base: This is the information the Assistant can search to improve its responses. It typically uses a vector database (like PgVector) to perform semantic searches over unstructured data, such as PDFs.
• Storage: This provides long-term memory for the Assistant. It uses a standard database to persist conversation history and user context, allowing the Assistant to remember previous interactions (e.g., answering 'What was my last message?').

The LLM OS design treats the LLM as a kernel process that coordinates various resources. The current implementation includes the following capabilities:

• Text Processing: Can read and generate text.
• Knowledge: Access to vast subject matter knowledge.
• Web Browsing: Can browse the internet.
• Software Integration: Can use existing software infrastructure like calculators, Python, and shell tools.
• Multi-Agent Communication: Can communicate with other LLMs.

Infrastructure Components:

• PgVector: Used as a Postgres-based long-term memory.
• Qdrant: Used as a vector database for the knowledge base (storing websites, PDFs, etc.).

The Mixture of Agents (MoA) approach uses multiple specialized agents to collaboratively complete a complex task. In this implementation, the workflow follows these stages:

1. Data Extraction: Specialized agents (e.g., MLB Researcher, MLB Batting Statistician, MLB Pitching Statistician) use tools to fetch raw game data.
2. Independent Generation: Multiple MLB Writer agents, each powered by a different language model (such as llama3-8b-8192, gemma2-9b-it, or mixtral-8x7b-32768), independently generate draft articles based on the collected data.
3. Synthesis: A final MLB Editor agent reviews all drafts and synthesizes the best elements into a single, polished final output.

This pattern allows for higher quality outputs by leveraging the diverse strengths of different LLMs.

Once the Personalized Agentic RAG application is running, you can interact with the agent to test memory and RAG capabilities:

Testing Memory (Personalization)

1. Enter a username to associate specific memory with that user.
2. Provide instructions to build memory, for example:
   • "I live in New York so always include a New York reference in the response"
   • "I like dogs so always include a dog pun in the response"
3. Verify the agent remembers these preferences in subsequent queries.

Testing Agentic Research

Ask complex questions that require tool usage, such as:

• "Compare nvidia and amd, use all the tools available"
• "Whats happening in france?"

Testing Agentic RAG (Knowledge Base)

1. Add a URL to the knowledge base (e.g., a blog post).
2. Ask specific questions based on that content, such as: "What does Sam Altman wish someone had told him?"

To use Phidata with LM Studio, you must have LM Studio installed and its Local Inference Server running. This setup allows you to use local LLMs via the OpenAI-compatible API provided by LM Studio.

Follow these steps to prepare your environment:

1. Create and activate a virtual environment.
2. Install the required libraries: phidata and openai.

Note: Since LM Studio provides an OpenAI-compatible endpoint, you will typically configure the OpenAI model class in Phidata to point to your local LM Studio server URL (usually http://localhost:1234/v1).

# 1. Create a virtual environment
python3 -m venv ~/.venvs/aienv
source ~/.venvs/aienv/bin/activate

# 2. Install libraries
pip install -U phidata openai

Install the required libraries for the Pinecone integration, including pinecone for vector database operations, pypdf for PDF processing, openai for LLM capabilities, and phidata for the agent framework.

pip install -U pinecone pypdf openai phidata

To run the Autonomous RAG example, follow these steps to set up your environment:

1. Create a virtual environment:

   python3 -m venv ~/.venvs/aienv
   source ~/.venvs/aienv/bin/activate

2. Set your API Key: Export your OpenAI API key as an environment variable:

   export OPENAI_API_KEY=***

3. Install dependencies: Install the required libraries using the provided requirements file:

   pip install -r cookbook/examples/auto_rag/requirements.txt

To use Ollama with Phidata, you must first install Ollama and pull the necessary models for embeddings and chat.

1. Install Ollama: Follow the official installation instructions for your OS at ollama.com.
2. Pull Embedding Model: Run ollama pull nomic-embed-text to prepare the embedding capabilities.
3. Pull Chat Model: Run ollama run openhermes to download and run the chat model.

Note: You can exit the chat model by typing /bye.

# Pull the embedding model
ollama pull nomic-embed-text

# Run the chat model
ollama run openhermes

To use Anthropic Claude with Phidata, follow these steps to set up your environment and install the necessary dependencies.

1. Create and activate a virtual environment:

   python3 -m venv ~/.venvs/aienv
   source ~/.venvs/aienv/bin/activate

2. Set your API Key: Export your Anthropic API key as an environment variable:

   export ANTHROPIC_API_KEY=xxx

3. Install required libraries: Install phidata along with the Anthropic client and various tool providers:

   pip install -U anthropic duckduckgo-search duckdb yfinance exa_py phidata

Execute the Lancedb Agent script using the following command:

python cookbook/integrations/lancedb/agent.py