Jinni

repository·main·Indexed 42 days ago

https://github.com/smat-dev/jinni

A tool designed to help LLMs efficiently read and understand project context by aggregating relevant code files into a concatenated view. It features a CLI and an MCP (Model Context Protocol) server with a `read_context` tool. Jinni supports sophisticated file filtering using `.contextfiles` and `.gitignore`-style syntax, allowing developers to include or exclude files and directories to optimize the context provided to AI agents. Version 0.4.0.

Tokens
9.3K
Snippets
21
Records
32
Agent score
52%

What's inside Jinni

  1. Configure file filtering with .contextfiles

    main

    Jinni uses .contextfiles to include or exclude files and directories during context generation. These files use .gitignore-style (gitwildmatch) syntax.

    Key Rules:

    • Inclusion: By default, files are included unless matched by an exclusion pattern.
    • Exclusion: Use ! at the start of a pattern to exclude a file or directory (e.g., !*.tmp).
    • Anchoring: A leading / anchors the pattern to the directory containing the .contextfiles.
    • Directories: A trailing / indicates the pattern applies only to directories (e.g., build/).
    • Recursion: ** matches zero or more directories (e.g., src/**/*.py).
    • Comments: Lines starting with # are ignored.

    Rules are applied dynamically: Jinni searches for .contextfiles and .gitignore files from the project root down to the current directory being processed. Rules from higher-level directories are applied before rules in the current directory.

    # Include all Python files in the src directory and subdirectories
    src/**/*.py
    
    # Include all Markdown files anywhere
    *.md
    
    # Exclude temporary files anywhere, even if they are .py or .md
    !*.tmp
    
    # Include the specific config file at the root of this context
    /config.yaml
    
    # Exclude the entire build directory relative to this file
    build/
  2. Run the Jinni MCP server using uvx

    main

    You can run the Jinni MCP server directly using uvx without performing a full installation of the jinni package. This is useful for quick execution or when you want to avoid managing the package in your local environment.

    Note that this package depends on the core jinni package for its functionality.

    uvx jinni-server
  3. Manage large context size limits

    main

    To prevent overwhelming an LLM or exceeding memory, Jinni enforces a total content size limit (default: 100MB). If the limit is exceeded, processing aborts with an error.

    How to configure:

    • Environment Variable: Set JINNI_MAX_SIZE_MB to change the global default.
    • CLI: Use the --size-limit-mb <int> flag.
    • MCP Server: Use the size_limit_mb argument in the read_context tool.
  4. Use ExclusionParser to convert natural language to pathspec patterns

    main

    The ExclusionParser class converts high-level exclusion directives (like keywords or scoped rules) into pathspec patterns used for file traversal. It supports four main types of exclusions:

    1. Global Exclusions (--not): Excludes files/directories based on keywords (e.g., tests, build, logs). If a keyword is recognized as a common module pattern, it expands to a set of standard patterns. If not recognized, it generates general patterns to match the keyword at various depths.
    2. Scoped Exclusions (--not-in): Excludes specific keywords only within a defined scope (e.g., src:legacy excludes legacy patterns only inside the src/ directory).
    3. File Pattern Exclusions (--not-files): Excludes specific file patterns (e.g., *.log). If no path separator is provided, the parser ensures the pattern matches at any depth.
    4. Keep Only (--keep-only): The most restrictive mode. It excludes everything (!*) and then explicitly includes only the specified modules.

    To use the parser during directory traversal, you must retain the ExclusionParser instance to call get_scoped_patterns for each directory encountered.

    from jinni.exclusion_parser import create_exclusion_patterns
    
    # Create patterns and the parser instance
    patterns, parser = create_exclusion_patterns(
        not_keywords=["tests", "logs"],
        not_in_scoped=["src:experimental"],
        not_files=["*.tmp"]
    )
    
    # 'patterns' contains the global and file-based exclusion pathspecs
    # 'parser' is used during traversal to handle scoped exclusions
  5. Use the Jinni MCP Server tools

    main

    Jinni provides an MCP (Model Context Protocol) server that exposes two primary tools to AI models (like Cursor or Claude): usage and read_context. These tools allow an AI to retrieve project documentation and read relevant code files to build context for development tasks.

    usage tool

    Returns the essential Jinni usage documentation, including rules and .contextfiles information. Use this when you are unsure of the rule syntax or how to configure Jinni.

    read_context tool

    Generates a concatenated view of relevant code files. It is designed to provide a 'canonical context' by focusing on files typically tracked in version control while respecting exclusions.

    Key behaviors for AI models:

    • Default Behavior: If rules is provided as an empty list [], Jinni uses sensible defaults (excluding .git, node_modules, etc.) and project-specific .contextfiles.
    • Targeting: Providing a specific list of files in targets is more efficient than reading the entire project.
    • Mandatory Arguments: project_root must be an absolute path.
    // Example tool call for read_context
    {
      "project_root": "/Users/username/projects/my-app",
      "targets": ["src/main.py", "README.md"],
      "rules": []
    }
  6. Use the Jinni CLI to process project files

    main

    Jinni is a CLI tool designed to process project files into a format suitable for LLM context. By default, it reads files in the specified paths, aggregates their content, and prints the result to stdout. The output is automatically copied to the clipboard unless the --no-copy flag is used.

    Basic Usage:

    • Process a specific directory: jinni ./my_project
    • Process multiple targets: jinni ./src ../docs/README.md
    • Write output to a file instead of stdout: jinni -o ctx.txt ./src
    • Specify a custom project root for path relativity: jinni -r /abs/path/root ./project
    # Process context for my_project
    jinni ./my_project
    
    # Write src context to a file
    jinni -o ctx.txt ./src
  7. Install the required 'pathspec' dependency

    main

    The jinni/config_system.py module depends on the pathspec library for pattern matching. If you encounter an ImportError, you must install it manually.

    pip install pathspec

    Or add it to your project's dependency file (e.g., requirements.txt).

  8. Use the jinni MCP Server tool: read_context

    main

    The jinni MCP Server provides the read_context tool to generate a concatenated view of relevant code files. This is useful for providing context to AI agents via the Model Context Protocol.

    Input Parameters:

    • project_root (string, required): Absolute path to the project root. Rule discovery and output paths are relative to this.
    • targets (JSON array of strings, required): Paths within project_root to process (e.g., ["src/main.py", "docs/"]). An empty list [] processes the entire project_root.
    • rules (JSON array of strings, required): Inline filtering rules (e.g., ["*.ts", "!*.test.ts"]). Note: If non-empty, these rules are used exclusively, ignoring built-in defaults and .contextfiles.
    • list_only (boolean, optional): If true, only returns file paths.
    • size_limit_mb (integer, optional): Overrides the default context size limit.
    • debug_explain (boolean, optional): Enables debug logging to stderr.
  9. Configure the size limit for context reading

    main

    Jinni enforces a maximum size limit on the generated context to prevent exceeding LLM token limits. You can control this limit in three ways, in order of precedence:

    1. Function Argument: Pass size_limit_mb directly to read_context().
    2. Environment Variable: Set JINNI_MAX_SIZE_MB in your environment.
    3. Default: If neither is provided, Jinni uses a default of 100 MB.

    If the limit is exceeded, a DetailedContextSizeError is raised, which includes a list of potential large files found relative to the project root to help you debug your .contextfiles configuration.

  10. Handle context size limit errors

    main

    When the total size of the processed files exceeds the configured limit, Jinni raises a DetailedContextSizeError. This error is designed to be actionable, providing:

    • The limit that was exceeded.
    • The specific file that caused the threshold to be crossed.
    • A list of large files found within the project root to assist in configuring exclusions.

    To resolve these errors, use a .contextfiles file to exclude large directories or files from the context.

  11. List files and token counts with Jinni

    main

    Jinni provides two modes for inspecting which files will be included in the context without outputting the full file content:

    1. List only paths: Use -l or --list-only to see a list of file paths that match your criteria. You can optionally include file sizes using -S or --size.
    2. List paths with token counts: Use -L or --list-token to see each file path followed by its token count (using tiktoken's cl100k_base encoding) and a final total sum.

    Example:

    # List files in current directory
    jinni -l .
    
    # List files with token counts
    jinni -L .
    jinni -L .
  12. Use the jinni CLI

    main

    The jinni CLI allows you to manually generate concatenated file contexts for your terminal or files.

    Usage: jinni [paths] [options]

    Arguments:

    • paths (positional, optional): Target directory or file paths. Defaults to ['.'].
    • --root <DIR> / -r <DIR>: Specify the project root for calculating relative output paths.
    • --output <file> / -o <file>: Write output to a file instead of stdout.
    • --list-only / -l: Only list file paths found.
    • --overrides <file>: Use rules from this file exclusively, ignoring defaults and .contextfiles.
    • --size-limit-mb <int> / -s <int>: Override the total content size limit.
    • --debug-explain: Enable detailed debug logging to stderr and jinni_debug.log.
    • --no-copy: Prevent automatically copying output to the clipboard (default is to copy).