DOCSEARCHbyAlgolia
Submit Docs

Esprima Documentation

repository·main·Indexed 27 days ago

https://github.com/jquery/esprima

A high-performance, standard-compliant ECMAScript parser written in JavaScript for lexical analysis (tokenization) and syntactic analysis of JavaScript code into an ESTree-compliant AST. Includes documentation for the esprima.tokenize(), esprima.parseScript(), and esprima.parseModule() functions, as well as the esparse and esvalidate CLI tools.

Tokens
11.4K
Snippets
38
Records
83
Agent score
92%

What's inside Esprima

  1. Understand the Esprima Syntax Tree Format

    main

    Esprima's syntax tree format is based on the Mozilla Parser API and follows the ESTree specification. Every node in the tree is a JavaScript object containing a type property that identifies its variant.

    If location information is enabled during parsing, nodes will also include range and loc properties for mapping back to the source code.

    interface Node {
  type: string;
  range?: [number, number];
  loc?: SourceLocation;
}

interface SourceLocation {
    start: Position;
    end: Position;
    source?: string | null;
}

interface Position {
    line: number;
    column: number;
}

  2. Build the documentation locally

    main

    To build the project documentation from source, you need to set up a Python virtual environment, install the required dependencies, and use make html within the docs directory. After the build completes, open _build/html/index.html in your web browser to view the documentation.

    ### Unix
```bash
virtualenv env
source env/bin/activate
pip install -r requirements.txt
cd docs
make html

    Windows

    virtualenv env
env\Script\activate
pip install -r requirements.txt
cd docs
make html

  6. Use Tolerant Mode to handle syntax errors

    main

    By default, Esprima throws an exception when it encounters invalid syntax. If you set the tolerant flag to true, the parser will attempt to continue parsing and return a syntax tree along with an errors array containing details about the issues encountered.

    Note: Tolerant mode is intended for a limited number of syntax error types and cannot robustly handle all invalid programs.

  7. Handle Hashbang/Shebang in source code

    main

    Esprima will throw an Error: Line 1: Unexpected token ILLEGAL if the source code contains a Unix shebang (e.g., #!/usr/bin/env node).

    To resolve this, you must remove or mask the first line before parsing.

    Option 1: Remove the line Use a regular expression to strip the line. Note that this changes the string length and offsets.

    Option 2: Preserve string length (Recommended for location mapping) Replace the shebang line with an equivalent number of whitespace characters. This ensures that subsequent character offsets and location data (when using { range: true }) remain accurate relative to the original source.

    var esprima = require('esprima')
var src = ['#!/usr/bin/env node', 'answer = 42'].join('\n')

// Replace shebang with spaces to preserve string length and offsets
src = src.replace(/(^#!.*)/, function(m) { return Array(m.length + 1).join(' ') });

esprima.parseScript(src, { range: true })

  8. Configure Esprima parsing options

    main

    You can customize the parser's output and behavior using a configuration object passed as the second argument to the parsing functions.

    NameTypeDefaultDescription
    jsxBooleanfalseSupport JSX syntax
    rangeBooleanfalseAnnotate each node with its index-based location (range property)
    locBooleanfalseAnnotate each node with its column and row-based location (loc property)
    tolerantBooleanfalseTolerate a few cases of syntax errors and return an errors array
    tokensBooleanfalseCollect every token
    commentBooleanfalseCollect every line and block comment

  9. Configure `esprima.tokenize()` options

    main

    You can pass a config object as the second argument to esprima.tokenize(input, config) to customize the output.

    NameTypeDefaultDescription
    rangeBooleanfalseAnnotate each token with its zero-based start and end location as an array [start, end].
    locBooleanfalseAnnotate each token with its column and row-based location object.
    commentBooleanfalseInclude every line and block comment in the output array.

  10. Implement syntax highlighting with Esprima tokens

    main

    You can use the range property of tokens to inject ANSI escape codes or HTML tags into a source string for syntax highlighting. A common pattern is to:

    1. Tokenize the source with { range: true }.
    2. Filter for specific token types (e.g., Identifier).
    3. Sort the tokens in reverse order by their start index.
    4. Iterate through the tokens and slice the original string to insert color codes at the range[0] and range[1] positions.

    Sorting in reverse order ensures that modifying the string length at one position does not invalidate the indices of subsequent tokens.

    const esprima = require('esprima');
const readline = require('readline');

const CYAN = '\x1b[36m';
const RESET = '\x1b[0m'
let source = '';

readline.createInterface({ input: process.stdin, terminal: false })
.on('line', line => { source += line + '\n' })
.on('close', () => {
    const tokens = esprima.tokenize(source, { range: true });
    const ids = tokens.filter(x => x.type === 'Identifier');
    // Sort reverse to prevent index shifting during string manipulation
    const markers = ids.sort((a, b) => { return b.range[0] - a.range[0] });
    markers.forEach(t => {
        const id = CYAN + t.value + RESET;
        const start = t.range[0];
        const end = t.range[1];
        source = source.slice(0, start) + id + source.slice(end);
    });
    console.log(source);
});