codefetch

The generated markdown follows a specific structure designed for AI context:

1. Project Structure: A tree view of the codebase (respecting .gitignore, .codefetchignore, and filters).
2. File Contents: Each file is presented with syntax highlighting and optional line numbers.
3. Token Count: A summary of total tokens used.
4. Metadata: Timestamps and configuration details.

Example Structure:

Project Structure:
├── src/
│   └── index.ts
└── package.json

src/index.ts:
```typescript
1 | export const main = () => {};

Project Structure: ├── src/ │ ├── index.ts │ └── utils/ │ └── helpers.ts └── package.json

src/index.ts:

1 | import { helper } from './utils/helpers';
2 | 
3 | export function main() {
4 |   console.log('Hello, world!');
5 | }

The playground showcases the following core capabilities of the codefetch-sdk:

• File Collection: Using collectFiles() with ignore patterns to gather source files.
• Token Counting: Using countTokens() with various encoders to manage context windows.
• Markdown Generation: Using generateMarkdown() to create structured documentation.
• Template Processing: Using processPromptTemplate() for dynamic prompt construction.
• Project Analysis: Using findProjectRoot() and other utility functions to navigate codebases.

When using format: 'json', the SDK returns a FetchResultImpl object. This object provides a structured view of the codebase instead of a flat string.

Properties:

• root: A FileNode representing the tree of files and directories.
• metadata: A FetchMetadata object containing aggregate statistics (e.g., token counts, file counts).

Methods:

• getFileByPath(path: string): FileNode | null: Retrieves a specific file node by its path.
• getAllFiles(): FileNode[]: Returns an array of all files in the collection.
• toMarkdown(): string: Generates a markdown representation from the structured tree.

Create a .codefetchignore file to prevent specific files or directories from being included in the documentation. It follows standard ignore patterns (similar to .gitignore).

Example .codefetchignore:

# Dependencies
node_modules/

# Build outputs
dist/
build/

# Test files
*.test.ts
__tests__/

By default, codefetch creates a codefetch/ directory in your project root to store all output files (unless using the --dry-run flag).

Best Practice: Add codefetch/ to your .gitignore file to prevent the generated markdown files from being committed to your repository.

When using --max-tokens, you can control how tokens are distributed across files using the --token-limiter option.

Strategies

• sequential: Processes files in order until the total token limit is reached. This is useful when you want complete content from the first files encountered.
• truncated (default): Distributes tokens evenly across all files, showing partial content from each file. This is useful for getting an overview of the entire codebase.

Usage

# Sequential mode
npx codefetch --max-tokens 500 --token-limiter sequential

# Truncated mode (default)
npx codefetch --max-tokens 500 --token-limiter truncated

The SDK includes a built-in in-memory cache. Re-using the same source URL within a single Node.js process or Worker isolate avoids redundant downloads and tokenization.

You can control caching using the noCache and cacheTTL options.

// Disable cache
await fetch({ source: './src', noCache: true });

// Custom TTL (seconds)
await fetch({ source: repoUrl, cacheTTL: 3600 });

The codefetch-sdk/worker includes built-in in-memory caching. This provides per-request memoization within the same isolate to reduce latency and GitHub API quota usage.

• Bypass Cache: Set noCache: true in your FetchOptions to force a fresh fetch.
• External Persistence: For cross-request persistence, combine the SDK with Cloudflare's Cache API or KV storage.

// Disable caching for a single request
await fetch({ source: repoUrl, noCache: true });

Codefetch uses two mechanisms to exclude files from analysis:

1. .gitignore: Respects your project's existing git ignore patterns.
2. .codefetchignore: Allows you to define additional patterns specific to codefetch analysis.

The .codefetchignore file follows the same syntax as .gitignore.

Codefetch also applies a set of default ignore patterns to exclude common files and directories (like build artifacts or dependencies) that are typically not useful for LLM analysis.

When calling fetch(), wrap the call in a try/catch block to handle common error scenarios. You can inspect error.message to provide specific responses to the client.

try {
  const result = await fetch({ source: repoUrl });
} catch (error) {
  if (error.message.includes('404')) {
    return new Response('Repository not found', { status: 404 });
  }
  if (error.message.includes('403')) {
    return new Response('Rate limit exceeded or auth required', { status: 403 });
  }
  if (error.message.includes('Invalid URL')) {
    return new Response('Invalid repository URL', { status: 400 });
  }
  console.error('Fetch error:', error);
  return new Response('Internal error', { status: 500 });
}

The open command generates your codebase as markdown, automatically copies it to your clipboard, and opens a specified AI chat interface in your browser with the model pre-selected. This is ideal for rapid code reviews or refactoring requests.

Use --chat-url to specify the service and --chat-model to select the specific model.

# Generate codebase, copy to clipboard, and open ChatGPT with GPT 5.1 Pro
npx codefetch open

# Open Gemini 3.0 instead
npx codefetch open --chat-url gemini.google.com --chat-model gemini-3.0

# Open Claude Sonnet
npx codefetch open --chat-url claude.ai --chat-model claude-3.5-sonnet

Convert local files into structured markdown with filtering and token management.

Common Local Tasks

• Filter by extension: Use -e or --extension (e.g., ts,tsx,js).
• Set token limits: Use --max-tokens to prevent context overflow.
• Specify output: Use -o for a filename or --output-path for a directory.
• Dry run: Use --dry-run to output to the console instead of a file.
• JSON format: Use --format json for programmatic access.

codefetch -e ts,tsx,js,jsx --max-tokens 100000 -o my-codebase.md