DOCSEARCHbyAlgolia
Submit Docs

Gittodoc

repository·main·Indexed 42 days ago

https://github.com/filiksyos/gittodoc

A tool that transforms Git repositories into documentation links optimized for AI coding assistants like Cursor. It includes the gitingest CLI tool (v0.1.4) to analyze codebases and create text dumps consisting of a summary, directory tree, and concatenated file contents for LLM context.

Tokens
3.8K
Snippets
18
Records
21
Agent score
54%

What's inside Gittodoc

  1. Overview of Gittodoc

    main
    Gittodoc is a tool designed to convert Git repositories into documentation links. These links are optimized to be fed into AI coding tools (such as Cursor) to provide them with repository context in a documentation-friendly format.

  4. Install and run Gittodoc via pip

    main

    To run Gittodoc locally using Python, ensure you have Python 3.7+ installed. You must clone the repository first, as the application is not published on PyPI. Install the necessary dependencies using the provided requirements.txt file.

    # Install dependencies
pip install -r requirements.txt

  5. Run Gittodoc with Docker Compose

    main

    You can run the Gittodoc application using Docker Compose. The setup exposes the application on port 8000. To ensure the application functions correctly (especially for S3 storage or GitHub integration), you must provide several environment variables in your host environment or a .env file.

    version: '3'

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      - GITINGEST_S3_BUCKET=${GITINGEST_S3_BUCKET}
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
      - AWS_REGION=${AWS_REGION}
      - GITHUB_PAT=${GITHUB_PAT}
    volumes:
      - ./src:/app  # Optional: Mount source code for development

  6. Configure allowed hosts via ALLOWED_HOSTS environment variable

    main

    The application uses TrustedHostMiddleware to restrict which host headers are accepted. You can control this by setting the ALLOWED_HOSTS environment variable as a comma-separated list of domains. If this variable is not set, the application defaults to a predefined list including localhost, 127.0.0.1, and various gittodoc.com or onrender.com subdomains.

    Note: If you are running behind a proxy, ensure your proxy is correctly configured to handle X-Forwarded-Host headers, otherwise TrustedHostMiddleware may trigger 400 errors.

    # Example: Setting allowed hosts in your environment
export ALLOWED_HOSTS="example.com,www.example.com,api.example.com"

  7. Format file system nodes with format_node()

    main

    The format_node function generates a structured representation of a codebase for ingestion. It produces three distinct parts: a summary, a directory tree structure, and the concatenated file contents.

    If the provided node is a directory, the function recursively processes all its children. If it is a single file, it summarizes that specific file.

    The returned tuple contains:

    1. Summary: Includes repository/directory metadata (name, commit, branch, subpath), file counts for directories, or file name and line counts for single files. It also includes an estimated token count (e.g., '1.2k', '1.2M') based on the cl100k_base encoding.
    2. Directory Structure: A visual tree-like string representation of the file hierarchy.
    3. File Contents: A single concatenated string containing the contents of all files within the node.
    from gitingest.output_formatters import format_node

# node: FileSystemNode, query: IngestionQuery
summary, tree, content = format_node(node, query)

  9. Check robots.txt for search engine crawlers

    main

    The application serves a robots.txt file at the /robots.txt endpoint to guide search engine crawlers. This file is served as a FileResponse from the static/ directory.

    GET /robots.txt HTTP/1.1
Host: your-domain.com

# Response
HTTP/1.1 200 OK
Content-Type: text/plain

User-agent: *
Disallow: /private/

  10. FileSystemNode: Represent and extract content from filesystem nodes

    main

    The FileSystemNode class represents a single entity in the filesystem (a file, directory, or symlink). It is used to track properties like name, path, size, and depth, and to manage hierarchical relationships via a children list.

    Key Capabilities

    • Content Extraction: Use the content_string property to get a formatted string suitable for LLM context. This includes a separator, the path (normalized with /), and the actual file content. For non-text files, it returns [Non-text file]. For Jupyter notebooks (.ipynb), it uses process_notebook to extract content.
    • Hierarchical Sorting: Directories can have their children sorted using sort_children(). This organizes nodes into a specific priority order:
      1. README.md (case-insensitive)
      2. Regular files
      3. Hidden files (starting with . )
      4. Regular directories
      5. Hidden directories All groups are sorted alphanumerically within themselves.

    Error Handling

    • Calling .content on a DIRECTORY node raises a ValueError.
    • Calling .sort_children() on a non-directory node raises a ValueError.
    • If a file cannot be decoded, content returns an error message instead of raising an exception.
    # Example of accessing content from a node
# Assuming 'node' is an instance of FileSystemNode
if node.type == FileSystemNodeType.FILE:
    print(node.content_string)

  11. Access API documentation via /api

    main

    The application provides a rendered HTML page containing API documentation at the /api/ or /api endpoints. This is served using Jinja2 templates.

    GET /api/ HTTP/1.1
Host: your-domain.com

# Response
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8

<html>...API Documentation Content...</html>

  12. Use the /health endpoint for server monitoring

    main

    The /health endpoint is a GET request used to verify that the server is running and healthy. It returns a JSON object containing the server status.

    GET /health HTTP/1.1
Host: your-domain.com

# Response
HTTP/1.1 200 OK
Content-Type: application/json

{
  "status": "healthy"
}