DOCSEARCHbyAlgolia
Submit Docs

Google Gemini Workshops

repository·main·Indexed 42 days ago

https://github.com/google-gemini/workshops

A collection of educational and experimental projects showcasing Google Gemini models. Includes a multi-agent dungeon adventure using CrewAI, an AI-powered interactive content platform (Artifacts), a fine-tuning pipeline for LDraw brick-building instructions (Bricks) using gemini-1.5-pro-002, a personalized AI video generation pipeline (Cameo) using ElevenLabs, and a Chess Companion utilizing Gemini Live and Vision APIs.

Tokens
199.7K
Snippets
506
Records
722
Agent score
53%

What's inside Gemini Workshops

  1. Overview of Factuality

    main
    Factuality is a Perplexity-style clone built using the CrewAI framework. It is designed to perform news summarization, citation of sources, and redaction of sensitive information. It serves as a demonstration of multi-agent orchestration for information retrieval and processing tasks.

  2. Overview of the AI Roast Battle Arena

    main

    The AI Roast Battle Arena is a multi-agent orchestration project that demonstrates how different LLMs can interact in an adversarial, comedic setting.

    Core Architecture

    • Multi-Agent Orchestration: Uses the CrewAI framework to manage distinct roles and complex interactions.
    • Roaster Agents: Configured (e.g., Google Gemini) for high-creativity, "no-holds-barred" comedic output using specific safety settings like BLOCK_NONE and temperature tuning.
    • Judge Agent: An impartial LLM (e.g., ChatGPT) that evaluates the quality of roasts based on wit, humor, and impact.
    • Contextual Memory: The system maintains a roast_history to ensure agents respond contextually to previous exchanges.
    • Structured Data: Uses Pydantic models to ensure LLM outputs are structured and reliable for programmatic use.

  3. Overview of Artifacts: AI-Powered Interactive Content Platform

    main

    Artifacts is a platform that explores AI-driven content creation, using Large Language Models (LLMs) to dynamically generate and refine interactive web experiences. It uses an iterative, conversational workflow to build complex web applications like physics simulations and games.

    Key capabilities include:

    • Dynamic Content Generation: AI agents create self-contained HTML documents containing all CSS, JavaScript, and images (as Base64 data URIs).
    • Iterative Refinement: Supports multi-turn conversations where the LLM's previous output is fed back as context for incremental construction.
    • Automated Demonstrations: A mechanism to cycle through various interactive artifacts (e.g., Tetris, Snake, physics simulations) for continuous display.
    • Single-File Portability: All artifacts are engineered as single HTML files for maximum portability and offline functionality.

  4. Overview of Wearable Gemini

    main
    Wearable Gemini is a project demonstrating how to embed Gemini AI capabilities into a wearable device. It uses a Raspberry Pi Zero W paired with an Adafruit Voice Bonnet to facilitate voice-based interaction. The system leverages Google Cloud's Speech-to-Text for processing spoken input and Text-to-Speech for generating spoken responses.

  5. Overview of TV Companion

    main

    TV Companion is an LLM-powered assistant designed for real-time multimodal interaction with home entertainment. It 'watches' movies and shows to provide contextual commentary, film trivia, and scene analysis. It can also control smart TVs via natural language voice commands.

    Key Capabilities:

    • Real-time Multimodal Input: Captures live audio/video via HDMI hardware or browser sources (e.g., Google Chrome).
    • Intelligent Scene Analysis: Detects scene changes and packages visual frames with transcribed dialogue.
    • Knowledge Grounding: Uses TMDB, Wikipedia, and film screenplays to provide factual commentary.
    • Voice-Controlled TV Actuation: Controls playback (play, pause, resume, search) via ADB.
    • Episodic Memory: Remembers viewing history and user preferences.
    • Flexible Commentary Modes: Supports automatic 'watching mode' (continuous commentary) or on-demand analysis.

  6. Overview of the AI Podcast Generator

    main
    The AI Podcast Generator is an automated pipeline for creating short, topical video podcasts. It transforms news headlines into audio-visual content by orchestrating Large Language Models (LLMs) and media processing libraries. The pipeline handles scripting, voice-overs, music integration, and final video assembly.

  7. Overview of the Bricks fine-tuning pipeline

    main
    The Bricks project is a workshop demonstration of an end-to-end pipeline for fine-tuning Gemini 1.5 Pro to generate LDraw (MPD) brick-building instructions from natural language descriptions. It automates the transition from simple user queries to precise, machine-readable 3D model files.

  9. View cost-comparison of Gemini vs. GPT vs. Llama 3

    main
    This project provides a hypothetical cost-comparison analysis for a high-scale application (99th percentile App Store app with 100,000 Daily Active Users). The comparison evaluates the costs of using Gemini, GPT, and Llama 3 models. Detailed analysis and data can be found in the hosted slides.

  10. Use Jyotish guru as a CrewAI agent for Kundali milan

    main

    The Kundali milan project implements a 'Jyotish guru' using the CrewAI agent framework to perform Kundali milan (astrological compatibility matching). It demonstrates how to leverage CrewAI agents to execute specialized domain tasks, such as analyzing astrological charts. An example of the output format can be found in the brad-pitt-angelina-jolie.pdf file within the repository.

    See example: ./brad-pitt-angelina-jolie.pdf

  11. Explore Workshop Projects by Category

    main

    The workshops are organized into several thematic categories to help you find relevant projects:

    AI/ML Applications & Agents

    Focuses on practical LLM applications, multi-agent systems, and problem-solving (e.g., adventure/, bricks/, factuality/, tv/).

    Games & Interactive Entertainment

    Focuses on AI interacting with game environments or simulating game theory (e.g., chess/, games/, smash/, waker/).

    Content & Media Generation

    Focuses on generating text, visual, audio, and video content (e.g., artifacts/, cameo/, doodles/, podcast/).

    Tools, Utilities & Core Concepts

    Focuses on development tools, fundamental AI concepts, and analytical insights (e.g., cost/, history/, scripts/).

    Hardware & Edge AI

    Focuses on integrating AI capabilities into physical hardware (e.g., wearable/).

  12. The Video Processing Pipeline

    main

    The complete pipeline for transforming raw video into a RAG-ready pedagogical tool consists of seven stages:

    1. Media Download (download-media.sh): Downloads audio, video, and YouTube metadata.
    2. Audio Transcription (transcribe-audio.ts): Generates audio segments with timestamps and confidence scores.
    3. Multimodal Frame Analysis (analyze-frames.ts): Performs visual, code, and audio analysis on 859+ segments.
    4. Segment-to-Concept Mapping (map-segments-to-concepts.ts): Maps segments to specific concepts using batched processing.
    5. Embedding Generation (embed-video-segments.ts): Creates 3072-dim embeddings with full YouTube provenance.
    6. RAG Integration: Implements semantic search and UI rendering with embedded players.
    7. Socratic Dialogue Integration: Enables video-grounded teaching with timestamp citations.