DOCSEARCHbyAlgolia
Submit Docs

bash-parser

repository·master·Indexed 42 days ago

https://github.com/vorpaljs/bash-parser

A standard compliant bash parser that converts bash source code into an Abstract Syntax Tree (AST). Built using a Jison grammar, it supports modes for bash, posix, and word-expansion, and provides configurable options for resolving shell aliases, environment variables, path globbing, and parameter expansions.

Tokens
3.1K
Snippets
9
Records
14
Agent score
50%

What's inside bash-parser

  1. Understand the AST output format

    master

    The ast object returned by parse() follows a specific structure. For a simple command like echo ciao, the output is a Script type containing an array of commands. Each command is represented as a SimpleCommand with a name (the command itself) and a suffix (the arguments).

    {
		type: "Script",
		commands: [
			{
				type: "SimpleCommand",
				name: {
					text: "echo",
					type: "Word"
				},
				suffix: [
					{
						text: "ciao",
							type: "Word"
					}
				]
				}
			]
	}

  2. Understand AST Node Types and Location Data

    master

    Every node in the Abstract Syntax Tree (AST) produced by bash-parser contains a type property that identifies its role.

    If you parse the source with the insertLOC option enabled, each node will also include a loc property. This property provides the exact coordinates of the node within the source string, including line numbers, column numbers, and character indices for both the start and end of the node.

    {
	type: 'WORD',
	value: 'ls',
	loc: {
		start: {
			row: Number,
			col: Number,
			char: 42
		},
		end: {
			row: Number,
			col: Number,
			char: 43
		}
	}
}

  3. Define a Mode plugin

    master

    To support different shell flavors, the parser uses mode plugins. A mode plugin is a module located in the src/modes folder that exports a ModePlugin object.

    This object must include:

    • init: A required factory function that takes the parentMode as an argument and returns a Mode object.
    • inherits (optional): A string specifying the name of the mode to inherit from. If provided, the init function receives the inherited mode as its argument, allowing the child mode to reuse parent features.
    type ModePlugin = {
	inherits?: string,
	init: (parentMode: Mode) => Mode
};

  5. Parse bash source code to an AST

    master

    You can parse bash source code by requiring bash-parser and calling the exported function with your bash string. The function returns an Abstract Syntax Tree (AST) representing the structure of the commands.

    const parse = require('bash-parser');
const ast = parse('echo ciao');

  6. Analyze Command and Function Nodes

    master

    Command

    Represents a builtin or external command. The name property is a Word representing the command name; it is optional if the node represents a bare assignment (e.g., VARNAME=42).

    {
	type: 'Command',
	name: ?Word,
	prefix: Array<AssignmentWord | Redirect>,
	suffix: Array<Word | Redirect>
}

    Function

    Represents a function definition. It includes the function name, an array of redirections applying to the whole body, and a body of type CompoundList.

    {
	type: 'Function',
	name: Name,
	redirections: Array<Redirect>,
	body: CompoundList
}

  7. Handle Word Expansions

    master

    Both Word and AssignmentWord nodes can contain an array of expansions.

    ArithmeticExpansion

    Represents an arithmetic expansion (e.g., $((...))). The internal arithmetic expression is parsed using the Babel parser. The loc property tracks the start and end indices within the parent Word text.

    {
	type: 'ArithmeticExpansion',
	expression: string,
	resolved: boolean,
	arithmeticAST: Object,
	type: 'ArithmeticExpansion',
	loc: {
		start: Number,
		end: Number
	}
}

    CommandExpansion

    Represents a command substitution (e.g., $(...)). The command is parsed recursively using bash-parser. The loc property tracks the start and end indices within the parent Word text.

    {
	type: 'CommandExpansion',
	command: string,
	resolved: boolean,
	commandAST: Object,
	loc: {
		start: Number,
		end: Number
	}
}

    ParameterExpansion

    Represents a parameter expansion (e.g., ${...}). The op and word properties represent the operator used and the right-hand Word for special parameters.

    {
	type: 'ParameterExpansion',
	parameter: string,
	kind: ?string,
	word: ?string,
	op: ?string,
	loc: {
		start: Number,
		end: Number
	}
}

  8. Structure a `Mode` object

    master

    The init function of a mode plugin must return a Mode object. This object defines how the shell flavor is parsed through the following properties:

    • tokenizer: A function that accepts options and utils, returning a function that takes shell source code and returns an Iterable<Token>.
    • lexerPhases: An array of LexerPhase transform functions applied sequentially to the tokens from the tokenizer.
    • phaseCatalog: A named map (dictionary) of all phases in the lexerPhases array, allowing child modes to access and reuse specific phases by name.
    • grammar: The compiled grammar function (typically generated from a Jison grammar).
    • grammarSource: A JSON object describing the grammar (Jison format), used by the mode-grammar-builder CLI and potentially reused by child modes.
    • astBuilder: An object containing methods to build the final AST. These methods are mixed into the Jison grammar and can be called directly from the EBNF source.
    export type Mode = {
	tokenizer: (options: Object, utils: Object) => (code: String) => Iterable<Token>,
	lexerPhases: Array<LexerPhase>,
	phaseCatalog: {[id:string]: LexerPhase}
	grammar: Object,
	grammarSource: Object,
	astBuilder: Object,
};

  9. Parse shell source code with parse()

    master

    The parse(code, [options]) function is the primary entry point for the module. It takes a string of shell source code and returns an object containing the Abstract Syntax Tree (AST) representing that code. You can provide an optional options object to configure how the parser handles various shell expansions and resolutions.

    const parse = require('bash-parser');
const ast = parse('echo $HOME');

  10. Analyze Control Flow Nodes (If, For, While, Until, Case)

    master

    If

    An if statement consisting of a clause (the condition), a then block, and an optional else block.

    {
	type: 'If',
	clause: CompoundList,
	then: CompoundList,
	else: CompoundList
}

    For

    Executes a sequence of commands for each member in a list.

    {
	type: 'For',
	name: Name,
	wordlist: Array<Word>,
	do: CompoundList
}

    While

    Executes a clause repeatedly as long as it returns a zero exit status.

    {
	type: 'While',
	clause: CompoundList,
	do: CompoundList
}

    Until

    Executes a clause repeatedly as long as it returns a non-zero exit status.

    {
	type: 'Until',
	clause: CompoundList,
	do: CompoundList
}

    Case

    Executes a CompoundList corresponding to the first matching pattern in a list of CaseItems.

    {
	type: 'Case',
	clause: Word,
	cases: Array<CaseItem>
}

  11. Implement `LexerPhase` functions

    master

    A LexerPhase is a transformation function used to enhance or alter the token stream produced by the tokenizer. These phases are applied in order to create a final token iterable that the grammar parser can consume.

    Each phase is a higher-order function that:

    1. Accepts options, an array of previousPhases, and a utils object.
    2. Returns a function that accepts an Iterable<Token> (from the previous phase) and returns a new Iterable<Token>.

    This allows for a pipeline architecture where each phase builds upon the work of the preceding ones.

    export type Token = Object;

type LexerPhase = (options: Object, previousPhases: Array<LexerPhase>,utils: Object) =>
	(tokens: Iterable<Token>) => Iterable<Token>;

  12. Configure parse() options for shell resolution

    master

    The options object passed to parse() allows you to provide callback functions that resolve shell-specific features during the parsing process. If a callback is not provided, the corresponding feature will not be resolved.

    Available Options

    OptionTypeDescription
    insertLOCBooleanIf true, includes line and column information in the AST. Defaults to false.
    resolveAlias(name: String) => StringCallback to resolve shell aliases. Return the resolved code or null if it doesn't exist.
    resolveEnv(name: String) => StringCallback to resolve environment variables. Return the value or null if undefined.
    resolvePath(text: String) => StringCallback to resolve path globbing. Return the expanded value.
    resolveHomeUser(username: String) => StringCallback to resolve tilde (~) expansion. If username is null, return the current user's home directory.
    resolveParameter(parameterAST: Object) => StringCallback to resolve parameter expansion. Return the result of the expansion.
    execCommand(cmdAST: Object) => StringCallback to resolve command substitution for a simple_command node. Returns the command output.
    execShellScript(scriptAST: Object) => StringCallback to resolve subshell statements by executing a complete_command node in a new shell process. Returns the output.
    runArithmeticExpression(arithmeticAST: Object) => StringCallback to resolve arithmetic expansion. The AST for these expressions is built using babylon. Returns the calculation result.