philschmid/gemini-samples

The examples/ directory contains Jupyter notebooks for various Gemini capabilities, including:

• SDK Integrations: Using Gemini with the OpenAI SDK, LangChain, or PydanticAI.
• Structured Data: Generating structured JSON outputs using Pydantic or dynamic meta prompts.
• Advanced Capabilities: Context caching (saving up to 75% on costs), Google Search integration, and code execution for data analysis.
• Multimodal & Media: Analyzing/transcribing YouTube videos, native image output (Gemini 2.0 Flash), and text-to-speech.
• Agentic & Protocol: Model Context Protocol (MCP) examples and CrewAI integration.
• File & Context: File editing, context URLs, and few-shot PDF extraction.

The repository includes specific examples for working with Gemma models:

• Gemma with GenAI SDK: Using Gemma 3 27B It with Google's GenAI API.
• Gemma Function Calling: Implementing function calling specifically for Gemma 3 27B.

For standalone scripts and JavaScript implementations:

Scripts (scripts/):

• Browser Interaction: gemini-browser-use.py for browser-based tasks.
• MCP Agents: Basic agent scripts and Pipedream integrations.
• Video & Image: Prompt optimization for Veo3, automatic video generation (vlogs), and image metadata generation.

JavaScript (javascript-examples/):

• Native Image Output: gemini-native-image-out.js for generating images directly with the Gemini 2.0 Flash experimental model.

The Gemini CLI supports several invocation modes:

• Interactive Mode (REPL): Start a conversational session by running gemini.
• Non-Interactive Mode: Pass a prompt directly using the -p flag. You can attach files using the @ syntax.

  gemini -p "Summarize the main points of the attached file. @./summary.txt"

- **Piping:** Pipe content from other commands into the CLI.
  ```bash
echo "Count to 10" | gemini

• Sandbox Mode: Run tools in a secure environment (requires Docker or Podman).

  gemini --sandbox -p "your prompt"

Custom commands allow you to define reusable prompts with arguments. You can store these in ~/.gemini/commands/ for global access or <project>/.gemini/commands/ for project-specific access.

Commands are invoked using the pattern /<command_name>:<subcommand> "<args>". The {{args}} placeholder in the TOML prompt field is replaced by the text you provide during invocation.

# Example: ~/.gemini/commands/test/gen.toml
# Invoked as: /test:gen "Create a test for the login button"

description = "Generates a unit test based on a description."
prompt = """
You are an expert test engineer. Based on the following requirement, please write a comprehensive unit test using the Jest testing framework.\n\nRequirement: {{args}}\n"""

To use the samples and guides in this repository, follow these steps:

1. Clone the repository:

   git clone https://github.com/philschmid/gemini-samples.git

2. Configure credentials: Create a .env file in the root directory and add your Gemini API key:

   GEMINI_API_KEY=your_api_key_here

3. Explore and Run: Browse the examples/, guides/, or javascript-examples/ directories to find relevant notebooks or scripts. You can run and modify these snippets for your own projects.

This repository provides several high-level guides for implementing advanced patterns with Gemini models:

• Function Calling Guide: Implementation details for function calling.
• ReAct Agent: Building agents using LangGraph and Gemini.
• Agentic Patterns: Overview of agentic patterns and their application with Gemini.
• Gemini with Memory: Integrating long-term memory into Gemini workflows.

You can connect the Gemini CLI to VS Code to enable workspace context (recent files, cursor position, selected text) and native diffing for code changes. Use the following commands within the CLI to manage the integration:

• /ide install: Sets up the VS Code integration.
• /ide enable: Connects the CLI to your active VS Code session.

Extensions allow you to add functionality like MCP (Model Context Protocol) servers, custom tools, and specific context files.

Setup:

1. Create a directory in <workspace>/.gemini/extensions/ or ~/.gemini/extensions/.
2. Include a gemini-extension.json file inside that directory to configure the extension.

Configuration Keys in gemini-extension.json:

• name: The name of the extension.
• version: The version string.
• mcpServers: An object defining MCP servers and their execution commands.
• contextFileName: Specifies the file used for extension context.
• excludeTools: An array of tool names to disable for this extension.

{
  "name": "my-extension",
  "version": "1.0.0",
  "mcpServers": {
    "my-server": {
      "command": "node my-server.js"
    }
  },
  "contextFileName": "GEMINI.md",
  "excludeTools": ["run_shell_command"]
}

In your prompt, use the @ symbol to reference files, directories, or media for context:

• Single file: > Explain this code. @./src/main.js
• Image: > Describe this. @./ux-mockup.png
• Directory (recursive): > Refactor this. @./src/

Note: The CLI respects .gitignore and .geminiignore files.

> Explain this code to me. @./src/main.js

Before using the CLI, you must authenticate using a Gemini API key from Google AI Studio. You can provide the key using one of two methods:

Method 1: Shell Environment Variable Set the GEMINI_API_KEY variable in your shell profile (e.g., ~/.bashrc or ~/.zshrc) to persist it across sessions:

export GEMINI_API_KEY="YOUR_GEMINI_API_KEY"

Method 2: Environment File Create a .env file in ~/.gemini/ (for global use) or ./.gemini/ (for project-specific use). The CLI loads this automatically:

# In .gemini/.env
GEMINI_API_KEY="YOUR_GEMINI_API_KEY"