DOCSEARCHbyAlgolia
Submit Docs

Hugging Face Agents Course

repository·main·Indexed 31 days ago

https://github.com/huggingface/agents-course

An educational course for building and evaluating AI agents. Topics include smolagents, LlamaIndex, LangGraph, Agentic RAG, fine-tuning for function-calling with LoRA, and agent observability using OpenTelemetry and Langfuse.

Tokens
43.9K
Snippets
88
Records
159
Agent score
98%

What's inside agents-course

  1. Overview of AI Agent Observability & Evaluation

    main

    Bonus Unit 2 focuses on advanced strategies for observing, evaluating, and improving AI agent performance. This is intended for developers moving from prototyping to production deployment.

    Key Learning Objectives

    • Instrumentation: Integrating observability tools via OpenTelemetry with the smolagents framework.
    • Metric Monitoring: Tracking performance indicators including token usage (costs), latency, and error traces.
    • Real-Time Evaluation: Implementing live evaluation techniques such as gathering user feedback and using LLM-as-a-judge.
    • Offline Analysis: Using benchmark datasets (e.g., GSM8K) to test and compare agent performance systematically.

  2. Overview of LlamaIndex for building agents

    main

    LlamaIndex is a toolkit designed for creating LLM-powered agents that operate over specific data using indexes and workflows. It is structured around four core concepts:

    • Components: The basic building blocks such as prompts, models, and databases that connect LlamaIndex to other libraries.
    • Tools: Specialized components that provide specific capabilities (e.g., searching, calculating, or accessing external services) which agents use to perform tasks.
    • Agents: Autonomous components that use tools and make decisions to coordinate complex goals.
    • Workflows: Step-by-step, event-driven, and async-first processes used to structure agentic behavior, either with or without explicit agents.

    Key advantages include a clear workflow system for composing logic, seamless integration with LlamaParse for advanced document parsing, a vast ecosystem of ready-to-use components, and access to LlamaHub, a registry of hundreds of components, agents, and tools.

  3. Overview of Fine-tuning for Function Calling (Bonus Unit 1)

    main

    Bonus Unit 1 covers the advanced topic of fine-tuning Large Language Models (LLMs) specifically for function calling. Unlike prompt-based approaches, fine-tuning trains the model to natively take actions and interpret observations during the training phase, leading to more robust AI agents.

    Key Learning Objectives:

    • Function Calling: Understanding how LLMs structure conversations to trigger tools.
    • LoRA (Low-Rank Adaptation): Using lightweight and efficient fine-tuning to reduce computational and storage overhead.
    • Thought → Act → Observe Cycle: Structuring how models decide on function calls, track steps, and interpret tool/API results.
    • Special Tokens: Implementing markers to distinguish between internal reasoning (Chain-of-Thought), outgoing function calls, and tool responses.

    Prerequisites:

    • Knowledge of fine-tuning LLMs with transformers.
    • Familiarity with using SFTTrainer for model fine-tuning.

  4. Introduction to LangGraph for complex LLM workflows

    main
    LangGraph is a framework designed for structuring and orchestrating complex, production-ready LLM workflows. Unlike simpler agent patterns, LangGraph provides fine-grained control over the flow of an agent, making it suitable for building robust and organized applications that require state management and specific execution paths.

  5. Overview of smolagents agent types

    main

    The smolagents library supports several specialized agent architectures depending on the required interaction pattern:

    • CodeAgents: The primary agent type. Instead of generating JSON or text, these agents produce and execute Python code to perform actions.
    • ToolCallingAgents: These agents rely on JSON or text blobs that the system parses and interprets to execute actions, rather than generating raw code.
    • Retrieval Agents: Agents designed to access knowledge bases using Retrieval-Augmented Generation (RAG) patterns, often leveraging vector stores and web search.
    • Vision Agents: Agents that incorporate Vision-Language Models (VLMs) to process visual information, enabling image-based reasoning and multimodal interactions.
    • Browser Agents: A specialized implementation (often using vision capabilities) that can navigate the web and extract information.

  6. Compare LangGraph and LangChain

    main

    While often used together, LangGraph and LangChain serve different purposes:

    • LangChain: Provides standard interfaces for interacting with models, retrieval components, and tool calls.
    • LangGraph: A framework for managing the control flow of applications. It uses LangChain components (like LLMs or tools) but focuses on the orchestration and state management of the overall workflow.

  7. Compare agentic frameworks: smolagents, LlamaIndex, and LangGraph

    main

    The course covers three primary frameworks for building agentic applications:

    FrameworkPrimary Use Case
    smolagentsA lightweight agents framework developed by Hugging Face.
    LlamaIndexEnd-to-end tooling designed to ship context-augmented AI agents to production.
    LangGraphFramework for building agents that allow for stateful orchestration.

  9. Understand the Observation phase in the Agent cycle

    main

    In the Thought-Action-Observation cycle, Observations are the signals an Agent receives from the environment after executing an action. They serve as the feedback loop that allows the agent to perceive the consequences of its actions and adapt its strategy.

    Key Functions of Observations:

    • Collect Feedback: Receives data or confirmation of action success/failure (e.g., API responses, error messages).
    • Append Results: Integrates new information into the agent's existing context/memory.
    • Adapt Strategy: Uses the updated context to refine subsequent thoughts and actions.

    Common Observation Types:

    TypeExamples
    System FeedbackError messages, success notifications, status codes
    Data ChangesDatabase updates, file system modifications, state changes
    Environmental DataSensor readings, system metrics, resource usage
    Response AnalysisAPI responses, query results, computation outputs
    Time-based EventsDeadlines reached, scheduled tasks completed

  10. Understand Large Language Model (LLM) architectures

    main

    LLMs are typically built on the Transformer architecture. There are three primary types of Transformers used in AI development:

    1. Encoders: Take text as input and output a dense representation (embedding). Used for text classification, semantic search, and Named Entity Recognition (e.g., BERT).
    2. Decoders: Focus on generating new tokens to complete a sequence, one token at a time. These are the most common models used for chatbots, text generation, and code generation (e.g., Llama, SmolLM2).
    3. Seq2Seq (Encoder–Decoder): Combine both to process an input sequence and generate an output sequence. Used for translation and summarization (e.g., T5, BART).

  11. Core concepts in smolagents

    main

    When building with smolagents, you will work with the following core components:

    • Tools: Functions that an LLM can use. They are the building blocks of agent behavior and can be implemented using the Tool class or the @tool decorator.
    • Multi-Agent Systems: The orchestration of multiple agents with different capabilities (e.g., combining a web search agent with a code execution agent) to create sophisticated solutions.
    • Memory Systems: Used primarily by retrieval agents to maintain conversation context while synthesizing information from multiple sources.

  12. Understand the concept of AI Tools

    main

    An AI Tool is a function provided to an LLM to allow it to perform actions beyond its native capabilities. Tools complement LLMs by providing access to up-to-date information (e.g., Web Search), specialized logic (e.g., Calculator), or external systems (e.g., API Interfaces).

    A complete tool definition for an LLM should include:

    • A textual description of what the function does.
    • Arguments with specific type hints.
    • Outputs with type hints (optional but recommended).

    LLMs cannot call tools directly; instead, the Agent identifies when a tool is needed based on the tool's description in the system prompt, generates a text-based invocation (e.g., call weather_tool('Paris')), and then executes the tool on the LLM's behalf.