DOCSEARCHbyAlgolia
Submit Docs

OSSInsight Documentation

repository·main·Indexed 24 days ago

https://github.com/pingcap/ossinsight

An analytics engine for the AI-native open source ecosystem that analyzes over 10 billion GitHub events. It provides developer and repository analytics, a natural language Data Explorer, and multiple API options including the OSSInsight MCP API for AI agents and the OSSInsight Public API (beta) for project insight data.

Tokens
92.7K
Snippets
131
Records
558
Agent score
82%

What's inside OSSInsight

  1. Overview of Agent-Native CLI patterns in 2026

    main

    The 2026 software landscape shows a shift toward providing structured CLI interfaces specifically for AI agents. These tools generally fall into three categories:

    1. Platform Builders: Infrastructure for creating new agent CLIs.

      • CLI-Anything: Provides the methodology and registry for GUI-to-CLI conversion.
      • jackwener/opencli: A universal hub to turn any website or tool into a CLI.

    2. Ecosystem Converters: Wrapping existing major software suites.

      • googleworkspace/cli: Dynamically built from Google Discovery Service to cover Drive, Gmail, etc.
      • larksuite/cli: Official CLI for Lark/Feishu (Messenger, Docs, etc.) with built-in AI Agent Skills.
      • agent-browser: Turns browser automation into a structured CLI command surface.

    3. Data Access Layers: Providing agents with unified read access to the web.

      • Agent-Reach: A unified CLI for social/content platforms (Twitter, Reddit, YouTube, etc.) that uses headless browsers to normalize content into structured JSON without requiring API keys.

  2. What is OSSInsight?

    main
    OSSInsight is an analytics engine that processes over 10 billion rows of GitHub event data to provide insights into the open source ecosystem. It allows users to track trends, developers, and repositories using metrics like commits, stars, forks, and contributors. It is specifically designed to be LLM-friendly, providing structured data for AI agents and researchers.

  3. Overview of the OSS Insight Workshop

    main

    The OSS Insight workshop is designed to help users learn how to use a new MySQL-compatible, scalable, and built-in analytical engine database. The workshop focuses on two main objectives:

    1. Learning the database: Mastering a scalable analytical engine that maintains MySQL compatibility.
    2. Application Building: Building small, interesting applications using the database.

    Participants can also obtain a free quota for database usage indefinitely.

  4. Future Roadmap: AI Agent Integration and MCP

    main

    OSSInsight is evolving to become a first-class data source for AI agents. Planned features include:

    • Agent-Friendly Endpoints: APIs designed to return structured, contextual data (rather than just HTML) specifically for agentic reasoning.
    • MCP Server: Exposing OSSInsight as a Model Context Protocol (MCP) server, allowing AI coding assistants to natively query repository statistics, community health, and ecosystem trends via tool calls.
    • AI-Generated Code Tracking: Analyzing commit patterns and PR metadata to surface signals regarding AI-assisted development within open-source projects.
    • Agent-Driven Exploration: Enhancing the Data Explorer to allow AI agents to autonomously chain multiple natural language queries to conduct complex research and generate narrative reports.

  5. Use the OSSInsight MCP API

    main

    The OSSInsight MCP API is a REST API designed for AI agents and Model Context Protocol (MCP) clients to programmatically access OSSInsight data.

    Base URL: https://ossinsight.io/api/mcp

    All requests use the GET method with query parameters. The API returns a consistent JSON response schema:

    • Success: { "ok": true, "data": ... }
    • Error: { "ok": false, "error": "Error message" }

    CORS is enabled for cross-origin access.

    // Success
{ "ok": true, "data": ... }

// Error
{ "ok": false, "error": "Error message" }

  6. What is DESIGN.md and how to use it

    main

    DESIGN.md is a convention introduced by Google Stitch to describe a design system in a plain markdown file. By placing a DESIGN.md file in your project root, AI coding agents can instantly understand your UI's visual identity, philosophy, and implementation rules without needing Figma exports or complex JSON schemas.

    A complete DESIGN.md implementation typically includes nine specific sections to capture the full design context:

    | Section | What It Captures |
|---------|-----------------|
| Visual Theme & Atmosphere | Mood, density, design philosophy |
| Color Palette & Roles | Semantic name + hex + functional role |
| Typography Rules | Font families, full hierarchy table |
| Component Stylings | Buttons, cards, inputs with states |
| Layout Principles | Spacing scale, grid, whitespace philosophy |
| Depth & Elevation | Shadow system, surface hierarchy |
| Do's and Don'ts | Design guardrails and anti-patterns |
| Responsive Behavior | Breakpoints, touch targets |
| Agent Prompt Guide | Quick color reference, ready-to-use prompts |

  7. Identify popular and active repositories

    main

    OSSInsight identifies top-performing repositories using two primary lenses:

    1. Popularity: Measured by the number of stars received within a specific timeframe (e.g., the top 50 projects by stars in 2022).
    2. Activity: Measured by the volume of collaborative events and repository growth over time (e.g., the most active repositories over a four-year period).

  8. Understand the OSSInsight Monorepo Architecture

    main

    OSSInsight has migrated from a fragmented multi-repo setup to a unified Turborepo monorepo using pnpm. This architecture consolidates the frontend, API, and documentation into a single codebase, enabling shared components and a consistent mental model.

    Core Components:

    • Main Site: A Next.js 16 App Router application (React 18/19, Tailwind) replacing the previous Docusaurus 2 SPA.
    • Documentation: Powered by Fumadocs instead of Docusaurus plugins.
    • API Layer: Integrated directly into the main app using Next.js Route Handlers, replacing the standalone Fastify server.
    • Widgets: Integrated directly into the main application rather than being a separate Next.js app.
    • Shared Packages: Includes @repo/site-shell which provides the header, navigation, and search across both the web app and the documentation site.
    • Pipelines: Managed as shared monorepo packages rather than standalone services.

  9. Analyze geographic distribution of developer behavior

    main

    OSSInsight analyzes how different countries interact with GitHub by querying event types (e.g., PushEvent, PullRequestEvent, IssueEvent) triggered by developers in specific regions.

    Key metrics include the proportion of different event types per country. For example:

    • US: High share of PullRequestReviewEvent.
    • Korea: High share of PushEvent.
    • Japan: High share of PullRequestEvent.
    • Germany: High share of IssueEvent and CommentEvent.
    • China: High share of WatchEvent and ForkEvent.

    Note: Data is typically sampled from developers who have a Location field populated in their GitHub profile.

  10. Explore repository contributor and maintainer analytics

    main

    OSSInsight provides deep-dive analytics for open-source repositories, allowing you to explore contributor distributions and analyze maintainer activity. You can use specific exploration IDs to view top contributors for a project or analyze individual maintainers to see their contribution time distribution and star history.

    Key features demonstrated in the analysis include:

    • Top Contributors: Visualizing the distribution of contributions within a repository.
    • Maintainer Analysis: Tracking a specific developer's contribution time distribution and project star history.
    • Commit History: Viewing the temporal progression of commits for a project.

  11. Analyze technical fields via GitHub Collections

    main

    OSS Insight provides curated GitHub Collections to help you find insights about monthly or historical rankings and trends within specific technical fields (e.g., Web Framework, Artificial Intelligence, Web3).

    Additionally, OSS Insight publishes deep-dive analytical reports (e.g., "Deep Insight Into Open Source Databases"). These reports include the SQL commands used to generate the charts, which you can run yourself on TiDB Cloud using the provided tutorials.

  12. Understand OSSInsight bot detection logic

    main

    OSSInsight uses specific heuristics to identify and filter out suspected bot accounts in its analysis.

    Star-based Bot Detection: A developer is suspected to be a bot if their star activity meets the following condition: $$\frac{\text{number of star events}}{\text{number of starred repositories}} \ge 2$$

    Event-based Filtering: Bot-triggered events are filtered out of many reports by matching regular expressions against event data to ensure human-centric insights.