DOCSEARCHbyAlgolia
Submit Docs

AI Distiller (aid)

repository·main·Indexed 41 days ago

https://github.com/janreges/ai-distiller

A high-performance code distillation tool that extracts public APIs and essential structures from large codebases to create optimized context for AI models. It reduces token usage and prevents hallucinations by providing accurate interface information. The project includes a CLI, an MCP server for Claude Code, and multiple parsing implementations using CGo and WASM (via Wazero) with tree-sitter.

Tokens
93.2K
Snippets
220
Records
361
Agent score
49%

What's inside AI Distiller

  1. What is AI Distiller (aid)?

    main
    AI Distiller (aid) is a code distillation tool designed to solve the context window limitations of AI models (like Claude, Gemini, or Cursor). It extracts essential information from a codebase—typically public APIs, data types, and interfaces—while discarding implementation details and non-public structures. This process reduces the source code volume to roughly 5-20% of its original size, allowing AI agents to understand the project's architecture and interfaces without exceeding their context limits or hallucinating incompatible code.

  2. Overview of Go Language Support in AI Distiller

    main
    AI Distiller supports Go (Golang) source code analysis by using the standard go/ast parser to build an Abstract Syntax Tree. It transforms Go code into a simplified, AI-friendly format that preserves essential structural information like the type system, method receivers, and visibility. This allows AI models to understand Go codebases more efficiently by focusing on the core logic and relationships rather than verbose syntax.

  3. Overview of Kotlin support in AI Distiller

    main
    AI Distiller uses the tree-sitter-kotlin parser to extract Kotlin code structures optimized for AI consumption. It is designed to preserve Kotlin's core principles, including null safety, immutability, and functional programming constructs. The distilled output maintains idiomatic syntax while stripping away implementation details to provide a clean structural view of the codebase.

  4. TypeScript Language Support in AI Distiller

    main
    AI Distiller provides comprehensive support for TypeScript codebases using the tree-sitter-typescript parser. It is designed to capture rich type information, structural patterns, and architectural relationships while minimizing token counts for efficient AI processing. It supports modern features like generics, decorators, and advanced type system constructs, preserving type safety contracts and generic constraints.

  5. Java Language Support Overview

    main

    AI Distiller supports Java 8+ codebases using the tree-sitter-java parser. It is designed to extract the essential structure of Java code—preserving type information, visibility modifiers, and relationships between classes, interfaces, and abstract classes—while reducing token counts for AI consumption.

    Compatibility

    • Minimum supported: Java 8
    • Recommended: Java 11+
    • Modern features: Partial support for Java 14+ (records, sealed classes, pattern matching)

    Supported Constructs

    • Foundational: Packages, Imports, Classes (abstract, final, nested), Interfaces (including sealed interfaces), Enums, and Records.
    • Object-Oriented: Methods (including constructors and overloading), Fields (all modifiers), Visibility (public, private, protected, package-private), Inheritance (extends, implements), Method Overriding (@Override), and Static Members.
    • Type System: Basic types (primitives/wrappers), Generics (type parameters and multiple bounds), Arrays, and Annotations (type/method/parameter annotations and @interface definitions).

    Unsupported/Limited Constructs

    • Lambda Expressions & Method References: Not parsed correctly.
    • Generics: Inheritance generics (e.g., extends Base<T>) are dropped.
    • Wildcards: ? extends T and ? super T are not supported.
    • Type Inference: The var keyword causes loss of type information.
    • Pattern Matching: Not currently recognized.

  6. Rust Language Support Overview

    main
    AI Distiller provides specialized support for Rust codebases using tree-sitter and line-based parsing. It is designed to capture the essential structure of Rust code—including ownership annotations, lifetime parameters, trait bounds, and async constructs—while optimizing the output for AI comprehension. The distillation process preserves Rust's type safety and memory safety guarantees by retaining critical type information while stripping away implementation noise.

  7. C++ Language Support Overview

    main
    AI Distiller provides comprehensive support for C++ codebases using the tree-sitter parser. It is designed to capture the essential structure of C++ code—including templates, multiple inheritance, operator overloading, and modern language features—while optimizing the output for AI consumption by preserving the powerful type system.

  8. Python Language Support Overview

    main
    AI Distiller provides comprehensive support for Python 3.x codebases using the tree-sitter-python parser. It is designed to extract the essential structure of Python code while preserving type information and API contracts, significantly reducing token counts for AI consumption while maintaining semantic meaning.

  9. Ruby Language Support Overview

    main
    AI Distiller provides comprehensive support for Ruby codebases using the tree-sitter-ruby parser. It is designed to capture the essential structure of Ruby code—including classes, modules, methods, and constants—while optimizing the output for AI consumption. The distillation process preserves Ruby's object-oriented design and metaprogramming capabilities.

  10. PHP Language Support Overview

    main
    AI Distiller provides comprehensive support for PHP 7.4+ codebases using the tree-sitter-php parser. It is designed to extract the essential structure of PHP code while preserving type information, visibility modifiers, and API contracts, which significantly reduces token counts for AI consumption while maintaining semantic meaning.

  11. Understand C# language support in AI Distiller

    main

    AI Distiller uses the tree-sitter-c-sharp parser to extract the structure of .NET codebases, supporting modern C# features up to C# 12. The distillation process aims to preserve strong typing, generic constraints, and object-oriented design while optimizing the output for AI consumption.

    Key capabilities include support for:

    • File-scoped namespaces (namespace Foo;)
    • Nullable reference types (#nullable enable/disable)
    • Init-only properties ({ get; init; })
    • User-defined operators
    • Tuples (named and unnamed)
    • Attributes on methods, classes, properties, and parameters
    • Enums with explicit base types
    • Async and extension methods

  12. Understand the AI Distiller Unified Test System structure

    main

    The Unified Test System automatically discovers and executes integration tests across all supported languages. Tests are organized in a hierarchical directory structure under testdata/.

    Directory Hierarchy

    • testdata/<language>/<NN_scenario_name>/
      • source.<ext>: The primary source file being tested (e.g., source.go, source.py).
      • helper.<ext>: (Optional) Additional files required for the test.
      • expected/: A subdirectory containing the expected output files.

    Naming Conventions

    • Scenario Directories: Use the format NN_descriptive_name (e.g., 01_simple_function). Numbers provide ordering but are optional.
    • Source Files: Must be named source.<ext> to be recognized as the primary test target.
    testdata/
├── python/
│   ├── 01_simple_function/
│   │   ├── source.py
│   │   ├── helper.py
│   │   └── expected/
│   │       ├── default.expected
│   │       └── test.implementation=0.expected