gemini-ai Ruby Gem

repository·main·Indexed 41 days ago

https://github.com/gbaptista/gemini-ai

A low-level Ruby gem for interacting with Google's Gemini AI models via Vertex AI, the Generative Language API, or AI Studio. It supports content generation, streaming via server-sent events, multimodal requests (image and video), embeddings, and function calling. Version 4.3.0 provides integration with Gemini 1.5 models, including JSON format responses and schema enforcement for Vertex AI.

Tokens
9.1K
Snippets
31
Records
35
Agent score
51%

What's inside gemini-ai

  1. Use Function Calling (Tools)

    main

    Function calling allows the model to request execution of specific tools. This is currently supported only by the vertex-ai-api service and the gemini-pro model.

    1. Define tools using function_declarations in the tools key, providing a name, description, and JSON Schema for parameters.
    2. The model returns a functionCall in its response.
    3. You must execute the function locally and then send the result back to the model using a message with role: 'function' and the functionResponse key.
    # 1. Define tool and request
    input = {
      tools: { function_declarations: [{ name: 'get_time', parameters: { ... } }] },
      contents: [{ role: 'user', parts: { text: 'What time is it?' } }]
    }
    result = client.stream_generate_content(input)
    
    # 2. Handle functionCall and prepare response
    # ... (execute local code) ...
    
    # 3. Send response back
    input[:contents] << result.dig(0, 'candidates', 0, 'content')
    input[:contents] << { role: 'function', parts: [{ functionResponse: { name: 'get_time', response: { content: '...' } } }] }
    
    # 4. Final request
    final_result = client.stream_generate_content(input)
  2. Implement Tool (Function) Calling

    main

    Tool calling allows the model to request execution of specific functions. You define these using tools with function_declarations (using JSON Schema for parameters).

    Workflow:

    1. Send request with tools definition.
    2. Model returns a functionCall in the response.
    3. Execute the function in your code.
    4. Send a new request containing the original conversation history plus a new message with role: 'function' and the functionResponse.
    # 1. Define tool and request
    input = {
      tools: {
        function_declarations: [
          {
            name: 'get_weather',
            description: 'Returns weather for a location',
            parameters: { type: 'object', properties: { location: { type: 'string' } } }
          }
        ]
      },
      contents: [{ role: 'user', parts: { text: 'What is the weather in London?' } }]
    }
    result = client.stream_generate_content(input)
    
    # 2. Handle functionCall and prepare response
    # (Extract functionCall from result, execute logic, then...)
    
    # 3. Send response back to model
    input[:contents] << result.dig(0, 'candidates', 0, 'content')
    input[:contents] << { 
      role: 'function', 
      parts: [{ 
        functionResponse: { 
          name: 'get_weather', 
          response: { name: 'get_weather', content: 'sunny' } 
        } 
      }] 
    }
    client.stream_generate_content(input)
  3. Understand the purpose of the gemini-ai gem

    main
    The gemini-ai gem is designed to provide low-level access to Google's Gemini models. It is intended for developers who want to build their own high-level abstractions on top of the Gemini API. If you require higher-level abstractions or more user-friendly tools, the project recommends considering Nano Bots.
  4. Initialize a Gemini client with different authentication methods

    main

    The Gemini.new constructor accepts a credentials hash to define the service and authentication method. You can use one of the following four patterns:

    1. Generative Language API (using an API Key)

    Best for AI Studio users. Set the service to 'generative-language-api' and provide an api_key.

    2. Vertex AI API (using a Service Account File)

    Best for Google Cloud users. Set the service to 'vertex-ai-api', provide a file_path to your JSON credentials, and specify a region.

    3. Vertex AI API (using Service Account File contents)

    Useful for environments where you prefer to pass credentials via environment variables. Set service to 'vertex-ai-api', provide file_contents (the raw string of the JSON), and specify a region.

    4. Vertex AI API (using Application Default Credentials)

    If you are running in a Google Cloud environment (like GCE, Cloud Run, or GKE) with configured ADC, simply provide the service and region.

    Note: You can also use the global region for a global endpoint.

    require 'gemini-ai'
    
    # 1. API Key
    client = Gemini.new(
      credentials: {
        service: 'generative-language-api',
        api_key: ENV['GOOGLE_API_KEY']
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
    
    # 2. Service Account File
    client = Gemini.new(
      credentials: {
        service: 'vertex-ai-api',
        file_path: 'google-credentials.json',
        region: 'us-east4'
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
    
    # 3. Service Account File Contents
    client = Gemini.new(
      credentials: {
        service: 'vertex-ai-api',
        file_contents: File.read('google-credentials.json'),
        region: 'us-east4'
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
    
    # 4. Application Default Credentials
    client = Gemini.new(
      credentials: {
        service: 'vertex-ai-api',
        region: 'us-east4'
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
    
    # Global Endpoint
    client = Gemini.new(credentials: { region: 'global' })
  5. Send multimodal requests (Image and Video)

    main

    To process images or videos, use a vision-capable model (e.g., gemini-pro-vision) and encode the media as Base64. Include the media in the parts array using the inline_data key with the appropriate mime_type.

    require 'base64'
    
    result = client.stream_generate_content(
      { contents: [
        { role: 'user', parts: [
          { text: 'Please describe this image.' },
          { inline_data: { 
            mime_type: 'image/jpeg', 
            data: Base64.strict_encode64(File.read('image.jpg')) 
          } }
        ] }
      ] }
    )
  6. Initialize the Gemini Client

    main

    To use the gem, create a new Gemini client by providing a credentials hash and an options hash. The credentials hash requires a service key, which can be either 'generative-language-api' or 'vertex-ai-api'.

    Depending on the service, you can authenticate using:

    • API Key: Use service: 'generative-language-api' and provide api_key.
    • Service Account File: Use service: 'vertex-ai-api' and provide file_path or file_contents.
    • Application Default Credentials: Use service: 'vertex-ai-api' and omit specific credential keys (requires environment setup).

    Common options include model (e.g., 'gemini-pro') and server_sent_events: true for streaming.

    require 'gemini-ai'
    
    # Example: Using an API Key
    client = Gemini.new(
      credentials: {
        service: 'generative-language-api',
        api_key: ENV['GOOGLE_API_KEY']
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
  7. Process multimodal inputs (Image and Video)

    main

    To use vision models (like gemini-pro-vision), you must encode your media as Base64 and provide the appropriate MIME type within the inline_data key of a part.

    require 'base64'
    
    # Image Example
    result = client.stream_generate_content(
      { contents: [
        { role: 'user', parts: [
          { text: 'Please describe this image.' },
          { inline_data: { 
            mime_type: 'image/jpeg', 
            data: Base64.strict_encode64(File.read('image.jpg')) 
          } }
        ] }
      ] }
    )
    
    # Video Example
    result = client.stream_generate_content(
      { contents: [
        { role: 'user', parts: [
          { text: 'Please describe this video.' },
          { inline_data: { 
            mime_type: 'video/mp4', 
            data: Base64.strict_encode64(File.read('video.mp4')) 
          } }
        ] }
      ] }
    )
  8. Request JSON format responses

    main

    To receive responses in JSON format, set response_mime_type to 'application/json' within the generation_config object.

    Note: As of current documentation, only the vertex-ai-api service and gemini models version 1.5 support this feature. For more control, you can also provide a response_schema (JSON Schema) to enforce a specific structure (supported by Gemini 1.5 Pro models).

    result = client.stream_generate_content(
      {
        contents: { role: 'user', parts: { text: 'List 3 random colors.' } },
        generation_config: {
          response_mime_type: 'application/json',
          response_schema: {
            type: 'object',
            properties: {
              colors: { type: 'array', items: { type: 'object', properties: { name: { type: 'string' } } } }
            }
          }
        }
      }
    )
  9. Configure API versions and global endpoints

    main

    Custom API Versions

    By default, the gem uses the v1 API version. To use a different version (like v1beta), pass the version key within the credentials hash.

    client = Gemini.new(
      credentials: {
        service: 'generative-language-api',
        api_key: ENV['GOOGLE_API_KEY'],
        version: 'v1beta'
      }
    )

    Global Endpoint

    To use the Google global endpoint for Vertex AI, set the region to 'global'.

    client = Gemini.new(credentials: { region: 'global' })
    # Example of setting a custom version
    client = Gemini.new(
      credentials: {
        service: 'vertex-ai-api',
        region: 'us-east4',
        version: 'v1beta'
      },
      options: { model: 'gemini-pro', server_sent_events: true }
    )
  10. Configure credentials for Vertex AI API (Application Default Credentials)

    main

    If you are using Application Default Credentials (ADC), the gem will automatically find them from your environment. This is common for local development using the gcloud CLI or running on Google Cloud infrastructure.

    To set up ADC locally, run:

    gcloud auth application-default login

    In your code, omit api_key and file_path:

    {
      service: 'vertex-ai-api',
      region: 'us-east4'
    }
  11. Configure credentials for Generative Language API (API Key)

    main

    To use the generative-language-api service, you need an API Key from Google Cloud Console or Google AI Studio. Ensure the Generative Language API is enabled in your Google Cloud project.

    Pass the credentials as follows:

    {
      service: 'generative-language-api',
      api_key: ENV['GOOGLE_API_KEY']
    }