jscpd Documentation

The cpd-tokenizer crate provides language-aware tokenization for over 200 programming languages. It generates token streams designed specifically for duplicate detection in copy/paste detection workflows.

Key features include:

• Generic tokenizer: Handles comments, strings, keywords, and punctuation.
• JS/TS/JSX/TSX tokenizer: Uses an Oxc-based implementation.
• Markdown tokenizer: Supports code fences, front matter, and embedded languages.
• SFC tokenizer: Supports Vue, Svelte, and Astro files.
• Embedded language detection: Enables cross-format tokenization for languages embedded within other formats.

The cpd-finder crate is a specialized component of the cpd project designed for file walking and clone detection orchestration. It manages directory traversal using ignore patterns (such as .gitignore and glob exclusions), dispatches tokenization based on file formats, performs clone detection and matching, enriches results with Git blame data to identify duplicate origins, and aggregates statistics.

Note: This crate is an internal component and is not intended for direct use. For the full command-line interface, use the jscpd crate.

The cpd-core crate provides the foundational data models and hashing utilities for the cpd copy/paste detector. It implements the core hash-based detection algorithm used to identify duplicated code.

Note: This crate is intended as a dependency for the cpd tool and is not designed for direct end-user consumption. For the full CLI experience, use the jscpd crate.

jscpd offers two distinct engines. Choose based on your performance and feature requirements.

Note: v5 is a drop-in replacement for v4 CLI usage (same options, config file, and reporters), but it lacks the Node.js programming API and persistent stores.

jscpd recognizes 223 language formats. These are categorized into three types:

1. Auto-detected formats (130): These are automatically recognized based on the file extension.
2. Registered formats (88): These formats have a grammar defined but no built-in extension mapping. To use these, you must explicitly activate them using the --formats-exts or --formats-names CLI flags.
3. Internal grammar dependencies (5): The formats clike, comments, hpkp, hsts, and keymap are internal dependencies used by other formats and are not intended for direct user selection.

Note: The url format is excluded from automatic detection.

The JSCPD Server supports the Model Context Protocol (MCP) via the POST /mcp endpoint. To use it with Claude Desktop, add the following configuration to your claude_desktop_config.json. Ensure the server is running (e.g., via npx jscpd-server . --port 3000) before starting Claude Desktop.

{
  "mcpServers": {
    "jscpd": {
      "type": "streamable-http",
      "url": "http://localhost:3000/mcp"
    }
  }
}

The JSCPD Server is a standalone REST API server for on-demand code duplication detection. It scans a codebase on startup and provides endpoints for checking snippets and retrieving statistics. It also supports the Model Context Protocol (MCP) for AI assistants.

Use the jscpd-server command to start the server. You can specify the target directory, port, host, and storage engine.

# Start server in current directory
jscpd-server

# Start server in specific directory
jscpd-server /path/to/project

# Start server on specific port
jscpd-server . --port 8080

# Start server with custom host
jscpd-server . --host localhost --port 3000

# Start server with persistent storage (recommended for production)
jscpd-server . --store leveldb

The jscpd-rs package provides a prebuilt Rust binary for the current platform. It is installed via npm in the tools/ directory. The binary is available at node_modules/.bin/jscpd-rs.

npm install

Add a check to your pre-commit hooks to prevent code with high duplication from being committed.

echo 'npx jscpd --threshold 5 --reporters console,silent .' > .husky/pre-commit

A prebuilt JAR is provided at tools/simian.jar. To update it:

1. Download the JAR from simian.quandarypeak.com.
2. Place the JAR at tools/simian.jar.

Requirements:

• Requires a Java runtime to execute.

java -jar tools/simian.jar -help

Use the official GitHub Action to run detection and upload SARIF reports to GitHub Code Scanning.

- uses: kucherenko/jscpd@master
  with:
    threshold: 5