webidl2.js

repository·main·Indexed 42 days ago

https://github.com/w3c/webidl2.js

A parser for the Web IDL (Interface Definition Language) specification used to define web APIs. It provides functionality to parse Web IDL files into an Abstract Syntax Tree (AST) via `parse()`, serialize ASTs back into strings via `write()`, and perform semantic validation using `validate()`. Version 24.5.0 supports both Node.js and browser environments, including a Tokeniser API and custom productions for extending the parser.

Tokens
4.5K
Snippets
18
Records
22
Agent score
53%

What's inside webidl2.js

  1. Overview of webidl2.js

    main
    webidl2.js is a parser for Web IDL, the language used to specify web APIs in an interoperable way. The library is designed to work in both Node.js and browser environments. You can also use the online checker to validate Web IDL files.
  2. Understand the WebIDL AST structure

    main

    The AST returned by parse() is a tree where the root is an array of definitions. Comments and whitespace are not included in the AST unless concrete: true is used (in which case an eof node is added).

    Common Node Types

    IDL Type

    Used for return types, arguments, etc. Fields include:

    • type: The usage context (e.g., attribute-type, return-type).
    • generic: The generic keyword (e.g., Promise, sequence).
    • idlType: The type name or array of subtypes.
    • nullable: Boolean.
    • union: Boolean.
    • extAttrs: Array of extended attributes.

    Interface

    • type: Always "interface".
    • name: The interface name.
    • partial: Boolean.
    • members: Array of attributes, operations, etc.
    • inheritance: Name of the parent interface or null.
    • extAttrs: Array of extended attributes.

    Dictionary

    • type: Always "dictionary".
    • name: The dictionary name.
    • members: Array of field nodes.
    • inheritance: The parent dictionary or null.

    Operation Member

    • type: "operation".
    • special: One of "getter", "setter", "deleter", "static", "stringifier", or "".
    • idlType: The return type object.
    • name: The operation name.
    • arguments: Array of argument objects.

    Attribute Member

    • type: "attribute".
    • special: One of "static", "stringifier", "inherit", or "".
    • readonly: Boolean.
    • idlType: The attribute type object.
    • name: The attribute name.
  3. Use parse, write, and validate in Node.js or the browser

    main

    WebIDL2 provides three primary functions for working with WebIDL:

    • parse(webidlString, options): Converts a WebIDL string into an Abstract Syntax Tree (AST).
    • write(ast, options): Converts an AST back into a WebIDL string. This is useful for programmatic code modification.
    • validate(ast): Performs semantic validation on an AST and returns an array of errors or warnings.

    Node.js Usage

    const { parse, write, validate } = require("webidl2");
    const tree = parse("string of WebIDL");
    const text = write(tree);
    const validation = validate(tree);

    Browser Usage

    Using a global WebIDL2 object:

    <script>
      const tree = WebIDL2.parse("string of WebIDL");
      const text = WebIDL2.write(tree);
      const validation = WebIDL2.validate(tree);
    </script>

    Or as an ES module:

    <script type="module">
      import { parse, write, validate } from "./webidl2/index.js";
      const tree = parse("string of WebIDL");
      const text = WebIDL2.write(tree);
      const validation = WebIDL2.validate(tree);
    </script>
    const { parse, write, validate } = require("webidl2");
    const tree = parse("string of WebIDL");
    const text = write(tree);
    const validation = validate(tree);
  4. Run tests for webidl2.js

    main
    The project uses mocha and expect.js for its test suite. If you have the environment set up, you can execute the tests from the root directory using the standard npm test command.
    npm test
  5. Implement custom productions in webidl2.js

    main

    You can extend webidl2.write() to support non-standard Web IDL syntax (e.g., specific browser implementations like Gecko) by providing an array of custom production functions via the productions option.

    Custom productions are called by the parser before the library's built-in productions. Each production function is a factory that receives a Tokeniser and returns an object. If you are using webidl2.write(), the returned object must implement a write(w: Writer) method.

    webidl2.write(ast, {
      productions: [
        // Custom production functions go here
      ]
    });
  6. Handle validation errors and autofixes

    main

    The validate() function returns an array of validation objects. Each object contains a message and a level ("error" or "warning").

    Some validation errors provide an autofix() function. You can call this function to automatically modify the AST to resolve the issue. After applying autofixes, you can call write(tree) to generate the corrected WebIDL.

    const validations = validate(tree);
    for (const validation of validations) {
      if (validation.autofix) {
        validation.autofix();
      }
    }
    write(tree); // Generates the fixed WebIDL
    const validations = validate(tree);
    for (const validation of validations) {
      if (validation.autofix) {
        validation.autofix();
      }
    }
    write(tree);
  7. Customize WebIDL output with write() templates

    main

    The write() function accepts an optional object containing a templates property. This allows you to intercept and modify how different parts of the WebIDL are stringified. Every property in the templates object is a function.

    Key template functions include:

    • wrap(items): Processes the final collection of strings. If absent, all items are raw strings.
    • trivia(t): Handles whitespace and comments.
    • name(escaped, { data, parent }): Handles container type identifiers (e.g., the Foo in interface Foo {}).
    • reference(escaped, unescaped): Handles type references (e.g., DOMString).
    • generic(name): Handles generic syntax keywords (e.g., sequence, Promise).
    • definition(content, { data, parent }): Handles the wrapped content of an IDL definition.
    • type(type): Called for each type definition (e.g., Document).

    Note: "Wrapped values" are raw strings unless a wrap() callback is provided.

    var result = WebIDL2.write(tree, {
      templates: {
        wrap: items => items.join(""),
        trivia: t => t,
        name: (escaped, { data, parent }) => escaped,
        reference: (escaped, unescaped) => escaped,
        generic: name => name,
        nameless: (keyword, { data, parent }) => keyword,
        type: type => type,
        inheritance: inh => inh,
        definition: (content, { data, parent }) => content,
        extendedAttribute: content => content,
        extendedAttributeReference: ref => ref
      }
    });
  8. Use the Tokeniser API in custom productions

    main

    When writing a custom production, you are provided with a Tokeniser object to navigate and consume the IDL string.

    Key Tokeniser Methods:

    • probeKind(type: TokenType): Checks if the current token matches a type (e.g., "identifier", "string", "whitespace") without consuming it.
    • probe(value): Checks if an inline token matches a specific value without consuming it.
    • consumeKind(...candidates: TokenType[]): Consumes the first token that matches any of the provided types. Returns null if no match is found.
    • consume(...candidates: string[]): Consumes an inline token if it matches any of the provided string values. Returns null if no match is found.
    • consumeIdentifier(value: string): Consumes an identifier token that matches the specific value.
    • error(message: string): Throws a WebIDLParseError at the current position.
    • unconsume(position: number): Seeks the position back to a previous point.
    type TokenType =
      | "decimal"
      | "integer"
      | "identifier"
      | "string"
      | "whitespace"
      | "comment"
      | "other";
    
    interface Tokeniser {
      position: number;
      constructor(idl: string);
      error(message: string): never;
      probeKind(type: TokenType): boolean;
      probe(value: any): boolean;
      consumeKind(...candidates: TokenType[]): any;
      consume(...candidates: string[]): any;
      consumeIdentifier(value: string): void;
      unconsume(position: number): void;
    }
  9. Configure parse() options

    main

    The parse() function accepts an optional configuration object with the following fields:

    • concrete: (Boolean) If true, the resulting AST will include an eof (End of File) node. This is necessary if you want the write() function to preserve trailing comments or whitespace.
    • productions: (Array) An array of custom production functions for extending the parser. See Custom productions.
    • sourceName: (String) A name for the source (e.g., a filename). This name is included in error and validation objects to indicate the origin of the issue.
    parse("string of WebIDL", {
      concrete: true,
      sourceName: "my-file.webidl"
    });
  10. Configure parser extensions

    main

    The parse function accepts an extensions object within the options parameter. This allows you to inject custom parsing logic for standard WebIDL constructs. This is useful when working with non-standard or extended WebIDL dialects.

    Supported extension keys include:

    • callbackInterface: Custom logic for parsing callback interfaces.
    • dictionary: Custom logic for parsing dictionaries.
    • interface: Custom logic for parsing interfaces.
    • mixin: Custom logic for parsing mixins.
    • namespace: Custom logic for parsing namespaces.

    Each extension key should map to a function that accepts a tokeniser and returns a parsed result.

    import { parse } from './webidl2.js';
    
    const options = {
      extensions: {
        interface: (tokeniser) => {
          // Custom interface parsing logic
        },
        dictionary: (tokeniser) => {
          // Custom dictionary parsing logic
        }
      }
    };
    
    const ast = parse(webidlString, options);
  11. Reference WebIDL syntax error properties

    main

    When parse() encounters a syntax error, it throws an exception with the following properties:

    • message: A descriptive error message including context.
    • bareMessage: The error message without context.
    • line: The line number where the error occurred.
    • sourceName: The source name provided during parse().
    • level: The severity ("error" or "warning").
    • ruleName: (Validation only) A specific identifier for the rule violated (e.g., attr-invalid-type, no-duplicate, constructor-member).
    • input: A short peek at the text at the error point.
    • tokens: The five tokens at the point of error as seen by the tokenizer.