GenAI (源内) Generative AI Utilization Platform

repository·main·Indexed 43 days ago

https://github.com/digital-go-jp/genai-ai-api

A generative AI platform by the Digital Agency of Japan providing secure AI microservices for government tasks. It features a RAG (Retrieval-Augmented Generation) pipeline including query expansion, knowledge base retrieval, answer generation, and reference formatting, with integrated support for AWS Bedrock and a hierarchical TOML configuration system.

Tokens
53.1K
Snippets
131
Records
207
Agent score
56%

What's inside GenAI

  1. Overview of the GenAI Azure Hosting Template

    main

    This repository provides a template for hosting and providing APIs for Hugging Face models (via vLLM) and Azure OpenAI models on Microsoft Azure. It is designed to efficiently manage inference workloads using a scalable architecture.

    Architecture Components

    Common Components:

    • Azure API Management (APIM): Acts as the single entry point. It handles external request management, including source IP restrictions, API key authentication, and request routing to the Application Gateway via private IP.
    • Azure Application Gateway (APGW): Performs load balancing to backend inference VMs. It uses a WAF policy to validate the x-apim-secret header added by APIM, ensuring only authorized APIM instances can communicate with it.

    vLLM-Compatible Model API Group:

    • Virtual Machine Scale Set (VMSS) / Virtual Machine (VM): Runs inference containers that retrieve models from Hugging Face. The VMSS configuration allows for flexible scaling by adding more VMs.
    • Azure Automation: Handles the periodic starting and stopping of VMs to manage costs.

    OpenAI API Group:

    • Azure Functions: A serverless environment that orchestrates Azure OpenAI APIs. It communicates with Azure OpenAI via private endpoints.
    • Azure OpenAI Service: Provides OpenAI APIs, including Chat Completions, Responses, and Code Interpreter-based data analysis (Excel/CSV).
  2. Overview of e-Laws Data Update Workflows

    main

    The e-Laws Data Update Workflows are automated GitHub Actions pipelines designed to periodically fetch the latest legal XML data from the e-Gov Laws API and reflect it in the BigQuery e_laws_search dataset.

    Processing Flow:

    1. Download latest legal XML data from e-Gov Laws API.
    2. Unzip the downloaded files.
    3. Create a date-stamped GCS bucket (e.g., {PROJECT_ID}-elaws-YYYYMMDD) with a 30-day automatic deletion lifecycle policy.
    4. Execute run_entire_pipeline.sh to load data into BigQuery:
      • Source Layer: Overwrites the source_laws_latest table (fixed name).
      • DWH & App Layers: Automatically updated.
    5. Clean up temporary files.
  3. Overview of GenAI (源内) AI Applications

    main

    GenAI (源内) is a generative AI utilization platform developed and operated by the Digital Agency of Japan. It provides an environment for government officials to use task-specific generative AI applications quickly, safely, and easily.

    The ecosystem consists of two main components:

    1. GenAI Web (源内 Web): The web application that users interact with directly.
    2. Administrative AI Apps (行政実務用 AI アプリ): Microservices that leverage generative AI.

    This repository contains implementations of administrative AI apps used in central government ministries. These apps can be built in environments independent of GenAI Web as long as they comply with the protocol used between GenAI Web and the microservices. Once built, they can be registered to GenAI via GUI operations.

  4. Overview of GENAI AI Applications

    main

    GENAI (Gennai) is a generative AI platform developed by the Digital Agency, Government of Japan, designed to provide secure and fast task-specific AI applications for government officials.

    The platform architecture consists of two primary components:

    • GENAI Web: The web application interface used by end users.
    • AI Applications for Government Operations: Microservices powered by generative AI that handle specific tasks.

    This repository contains a subset of the AI applications deployed in central government ministries. Developers can build AI applications in environments independent of GENAI Web, provided they comply with the communication protocol between the application and GENAI Web. Once compliant, applications can be registered to GENAI via GUI operations.

  5. Overview of Lawsy-Custom-BQ

    main

    Lawsy-Custom-BQ is a serverless API system designed to receive questions regarding Japanese laws and dynamically generate optimized reports using AI. It combines BigQuery ML vector search with Google Gemini to perform searches and answer generation against Japanese legal data (e-Gov).

    Core Capabilities

    • Legal Search: High-speed searching of relevant laws using BigQuery ML vector search.
    • Report Generation: Automatically selects one of six optimal structures based on query intent to generate comprehensive Markdown reports in a single AI call.
    • Source Management: Organizes and displays only the reference information actually cited in the response.

    Technical Stack

    • Backend: Python (Google Cloud Functions)
    • AI Model: Google Vertex AI (Gemini 2.5 Flash)
    • Data Source: BigQuery ML Vector Search + Gemini Web Grounding
    • Infrastructure: Google Cloud (API Gateway, Cloud Functions, BigQuery ML)
    • IaC: Terraform
  6. Overview of the genai-azure architecture

    main

    This repository provides templates for efficiently hosting Hugging Face models (via vLLM) and Azure OpenAI models on Microsoft Azure and providing them as APIs.

    Core Architecture Components

    Common Components:

    • Azure API Management (APIM): Acts as the single entry point for external requests. Handles IP restrictions, API key authentication, and request routing. It communicates with the Application Gateway via private IP over HTTP.
    • Azure Application Gateway (APGW): Receives requests from APIM and load balances them to backend inference VMs. It uses a security mechanism where APIM adds an x-apim-secret header, which APGW validates via WAF policy to ensure only authorized APIM requests are accepted.

    vLLM Model API Group:

    • Virtual Machine Scale Set (VMSS) / Virtual Machine (VM): Hosts inference containers that fetch models from Hugging Face and serve them via REST API. The use of VMSS allows for flexible scaling by adding more VMs.
    • Azure Automation: Handles the scheduled starting and stopping of VMs to manage costs.

    OpenAI API Group:

    • Azure Functions: A serverless environment that orchestrates the execution of Azure OpenAI APIs. Communication with Azure OpenAI is performed via private endpoints for secure, closed-network access.
    • Azure OpenAI Service: Provides OpenAI APIs, including Chat Completions/Responses APIs and Code Interpreter APIs for data analysis (Excel/CSV).
  7. System Overview: Azure Functions Code Interpreter API

    main

    This API enables data analysis using Azure OpenAI's Code Interpreter capabilities via Azure Functions.

    Key Features:

    • Upload Excel files (.xlsx, .csv).
    • Perform data analysis using Azure OpenAI Code Interpreter.
    • Automatic generation of graphs (with support for Japanese fonts).
    • Returns both analysis results and generated graphs.

    Architecture: Client $\rightarrow$ Azure Functions $\rightarrow$ Azure OpenAI (Code Interpreter)

  8. Overview of APIM Policy Customization

    main

    Azure API Management (APIM) policies allow you to transform requests and responses between a simplified client-facing API and a complex backend API (such as vLLM's OpenAI-compatible API).

    This is useful for:

    • Simplifying Requests: Hiding model-specific prompt formats or special tokens from the client.
    • Reducing Response Verbosity: Stripping away unnecessary metadata so the client only receives essential information.
    • Improving Usability: Providing a clean, purpose-built API specification for end-users.
  9. Safety Settings and Guardrails in Lawsy-Custom-BQ

    main

    Because Japanese legislation often contains text describing prohibited or discriminatory acts, standard AI safety filters (which might block such content) are set to BLOCK_NONE for the Gemini API.

    To prevent misuse and ensure quality, the system implements guardrails through System Prompts (defined in modules/api/functions/src/prompts.py) rather than API-level safety settings. These prompts enforce:

    • Role Limitation: Restricting the model to a specialized legal query responder.
    • Accuracy: Instructing the model not to speculate on uncertain content.
    • Mandatory Citations: Requiring structured output with article numbers and source links.
  10. Define configuration data structures with config_types.py

    main

    To ensure type safety and IDE autocompletion, use config_types.py to define the structure of your configuration items using Python dataclasses and Enum. This prevents errors caused by typos in dictionary keys and provides explicit type validation for configuration parameters.

    from dataclasses import dataclass
    from enum import Enum
    
    class ModelProvider(Enum):
        ANTHROPIC = "anthropic"
        AMAZON = "amazon"
    
    @dataclass
    class AnswerGenerationConfig:
        model_id: str
        temperature: float
        max_tokens: int
        system_prompt: str
  11. Compare `utils/` and `services/` directories

    main

    Understanding the distinction between utilities and services helps maintain a clean architecture:

    Aspectutils/services/
    DependenciesPrimarily standard librariesAWS SDK, external APIs
    Domain KnowledgeNot requiredRequired (e.g., RAG, Bedrock)
    StateStatelessMay hold state
    ReusabilityReusable across projectsProject-specific
    TestingEasy, no mocks requiredRequires mocking

    Example:

    • format_timestamp(timestamp) $\rightarrow$ utils/ (Generic, stateless)
    • track_bedrock_usage(model_id, tokens) $\rightarrow$ services/ (Bedrock-specific, stateful)