DeepWiki-Open (Grok-Wiki)

repository·main·Indexed 12 days ago

https://github.com/asyncfuncai/deepwiki-open

An automated documentation engine that transforms GitHub, GitLab, or BitBucket repositories into interactive visual wikis. It features a FastAPI-based streaming API, RAG-powered code analysis, and support for both cloud providers (Google Gemini, OpenAI, AWS Bedrock) and local LLMs via Ollama using models like qwen3:1.7b and nomic-embed-text.

Tokens
26.6K
Snippets
73
Records
120
Agent score
87%

What's inside DeepWiki-Open

  1. Overview of DeepWiki-Open

    main

    DeepWiki-Open (Grok-Wiki) is an automated tool that generates interactive wikis for any GitHub, GitLab, or BitBucket repository. It performs code structure analysis, generates full documentation, builds visual Mermaid diagrams, and creates a 'codemap' for step-by-step code walkthroughs.

    Key Features:

    • Instant Documentation: Converts repositories into wikis quickly.
    • Private Repository Support: Secure access via personal access tokens.
    • AI-Powered Analysis: Understands code relationships using AI.
    • Visual Diagrams: Automatically generates Mermaid diagrams for architecture and data flows.
    • RAG-based Chat: A "Ask" function to interact with the repository using Retrieval-Augmented Generation.
    • DeepResearch: Multi-step research for deep analysis of complex topics.
    • Multi-Provider Support: Works with Google Gemini, OpenAI, OpenRouter, Azure OpenAI, and local Ollama models.
  2. What is DeepWiki-Open (Grok-Wiki)

    main

    DeepWiki-Open (also known as Grok-Wiki) is a tool designed to automatically generate interactive, high-quality wikis for GitHub, GitLab, or BitBucket repositories.

    When provided with a repository name, the tool performs the following workflow:

    1. Code Structure Analysis: Analyzes the repository's architecture.
    2. Documentation Generation: Creates comprehensive technical documentation.
    3. Visual Diagramming: Generates diagrams to explain system logic and flow.
    4. Wiki Organization: Structures the content into a navigable wiki format.
    5. Codemap Generation: Creates code-centric guided tours to help users navigate the codebase.
  3. Project Structure Overview

    main

    The DeepWiki-Open repository is organized into a backend API and a frontend web application:

    • api/: The backend API server.
      • main.py: API entry point.
      • api.py: FastAPI implementation.
      • rag.py: Retrieval Augmented Generation logic.
      • data_pipeline.py: Data processing utilities.
      • requirements.txt: Python dependencies.
    • src/: The Next.js frontend application.
      • app/: Next.js application directory.
      • components/: React components (e.g., Mermaid.tsx for diagram rendering).
    • public/: Static assets.
    • .env: Environment variable configuration file.
  4. Project Structure

    main

    Understanding the directory layout for development or customization:

    • api/: The backend API server (FastAPI).
      • main.py: API entry point.
      • api.py: FastAPI implementation.
      • rag.py: RAG (Retrieval-Augmented Generation) logic.
      • data_pipeline.py: Data processing utilities.
    • src/: The Next.js client application.
      • app/: Next.js application directory.
      • components/: React components (e.g., Mermaid.tsx for diagram rendering).
    • public/: Static assets.
    • .env: Environment variable configuration.
  5. Project Structure of DeepWiki-Open

    main

    Understanding the directory layout helps in customizing the backend or frontend:

    • api/: Backend API server containing main.py (entry point), api.py (FastAPI implementation), rag.py (RAG logic), and data_pipeline.py (data processing).
    • src/: Frontend Next.js application, including app/page.tsx and components/Mermaid.tsx for diagram rendering.
    • public/: Static assets.
    • .env: Required environment variable configuration.
    deepwiki/
    ├── api/                  # Backend API server
    │   ├── main.py           # API entry point
    │   ├── api.py            # FastAPI implementation
    │   ├── rag.py            # Retrieval-Augmented Generation
    │   ├── data_pipeline.py  # Data processing utilities
    │   └── requirements.txt  # Python dependencies
    │
    ├── src/                  # Frontend Next.js app
    │   ├── app/              # Next.js app directory
    │   │   └── page.tsx      # Main application page
    │   └── components/       # React components
    │       └── Mermaid.tsx   # Mermaid diagram renderer
    │
    ├── public/               # Static assets
    ├── package.json          # JavaScript dependencies
    └── .env                  # Environment variables (must be created)
  6. Use Ask and DeepResearch features

    main

    DeepWiki provides two primary ways to interact with your repository:

    Ask (RAG-based Chat)

    Uses Retrieval Augmented Generation (RAG) to chat with your repository. It provides context-aware answers based on actual code snippets, supports real-time streaming, and maintains conversation history.

    DeepResearch (Multi-turn Research)

    Performs an in-depth, multi-turn research process for complex topics. It follows a structured process:

    1. Research Plan: Outlines the approach and initial findings.
    2. Research Update: Adds new insights based on previous iterations.
    3. Final Conclusion: Synthesizes all iterations into a comprehensive answer.

    How to use DeepResearch: Before submitting a question in the Ask interface, toggle the "Deep Research" switch to ON. The AI may iterate up to 5 times automatically to complete the research.

  7. How DeepWiki-Open works

    main

    DeepWiki-Open automates the creation of interactive wikis for GitHub, GitLab, or Bitbucket repositories through the following workflow:

    1. Repository Access: Clones the repository. For private repositories, users must provide a Personal Access Token.
    2. Analysis: Analyzes the code structure and creates embeddings for intelligent retrieval.
    3. AI Generation: Uses a selected AI model (Google Gemini, OpenAI, OpenRouter, Azure OpenAI, or local Ollama) to generate context-aware documentation.
    4. Visualization: Automatically creates Mermaid diagrams to visualize architecture and data flows.
    5. Organization: Compiles everything into a structured, navigable wiki.
    6. Interaction: Enables a "Ask" feature (RAG-based Q&A) and "DeepResearch" for complex topic exploration.
  8. Configure LLM providers and models

    main

    DeepWiki supports multiple model providers. You can switch between them by providing the corresponding API keys in your environment variables.

    Supported Providers:

    • Google: Default model gemini-2.5-flash. Also supports gemini-2.5-flash-lite, gemini-2.5-pro, etc.
    • OpenAI: Default model gpt-5-nano. Also supports gpt-5, 4o, etc.
    • OpenRouter: Access to various models (Claude, Llama, Mistral, etc.) via a single API.
    • Azure OpenAI: Default model gpt-4o.
    • Ollama: Local open-source models (e.g., llama3).

    To use a custom OpenAI-compatible endpoint (like Alibaba Qwen) for embeddings, replace api/config/embedder.json with embedder_openai_compatible.json and set OPENAI_BASE_URL.

    # Example for OpenAI-compatible embedding providers
    OPENAI_API_KEY=your_key
    OPENAI_BASE_URL=https://compatible-endpoint/v1
  9. Use the Question and Deep Research features

    main

    DeepWiki-Open provides two primary modes for interacting with your repository:

    Questioning (RAG-powered)

    Use this for standard chat interactions. It uses Retrieval-Augmented Generation (RAG) to:

    • Provide context-aware answers based on actual repository code.
    • Retrieve relevant code snippets as evidence for answers.
    • Support real-time streaming of responses.
    • Maintain conversation history for consistent interactions.

    Deep Research

    Use this for complex topics that require multi-turn analysis. To use it, toggle the "詳細調査" (Deep Research) switch in the question interface before sending your query.

    Deep Research follows a structured process:

    1. Research Plan: Outlines the approach and initial findings.
    2. Research Updates: Evolves the investigation by adding new insights.
    3. Final Conclusion: Provides a comprehensive answer based on all iterations.

    The AI will automatically continue research for up to 5 iterations until a conclusion is reached.

  10. Use Deep Research and Questioning features

    main

    DeepWiki provides two primary ways to interact with your repositories:

    Questioning (RAG-driven)

    Allows you to chat with your repository using Retrieval-Augmented Generation (RAG). It provides context-aware responses based on actual code, supports real-time streaming, and maintains conversation history.

    Deep Research

    An advanced multi-turn research process that performs deep investigations into complex topics. It follows a structured plan:

    1. Research Plan: Outlines the method and initial findings.
    2. Research Updates: Adds new insights based on previous iterations.
    3. Final Conclusion: Provides a comprehensive answer based on all iterations.

    Deep Research can automatically iterate up to 5 times to reach a conclusion. To use it, toggle the "Deep Research" switch in the questioning interface before submitting your question.

  11. Use Questions and DeepResearch features

    main

    DeepWiki provides two primary ways to interact with your repository:

    Questions (RAG)

    Allows you to chat with your repository using Retrieval Augmented Generation (RAG). It provides contextual answers based on actual code, uses real-time streaming, and maintains conversation history.

    DeepResearch

    An advanced, multi-step research process for complex topics. It follows a structured plan and automatically continues research for up to 5 iterations until a conclusion is reached.

    Research Stages:

    1. Research Plan: Describes the approach and initial findings.
    2. Research Updates: Iterative insights built on previous steps.
    3. Final Conclusion: A comprehensive response based on all iterations.

    To use DeepResearch: Toggle the "Pesquisa Aprofundada" (Deep Research) switch in the Questions interface before sending your query.

  12. How Question and Deep Research functions work

    main

    DeepWiki-Open provides two primary modes for interacting with your repository:

    Question Function (RAG-based)

    Allows you to chat with your repository using Retrieval-Augmented Generation (RAG). Key features include:

    • Context-Aware Responses: Answers are based on the actual code in your repository.
    • RAG-Powered: The system retrieves relevant code snippets to ground its answers.
    • Real-time Streaming: Responses are streamed as they are generated.
    • Conversation History: Maintains context across multiple questions for coherent interactions.

    Deep Research Function

    Designed for complex topics, this mode uses a multi-turn research process that automatically iterates until a conclusion is reached (up to 5 iterations). The process follows these stages:

    1. Research Plan: Describes the approach and initial findings.
    2. Research Updates: Develops previous iterations with new perspectives.
    3. Final Conclusion: Provides a comprehensive response based on all iterations.

    To use Deep Research: Toggle the "Investigación Profunda" (Deep Research) switch in the Question interface before submitting your query.