swift-syntax

SwiftParser follows several core design principles:

• Resilient: Recovers from syntax errors by using unexpected nodes and missing tokens instead of failing.
• Efficient: Aiming for performance comparable to the C++ parser implementation.
• Source-preserving: Maintains all "trivia" (whitespace, comments, etc.) so that the syntax tree can be rendered back into byte-for-byte identical source text.
• Minimal context: Requires only minimal context (such as dialect support for regex literals) and can be invoked on any major grammar production (e.g., a full file, a type, or an expression).
• Incremental: Supports incremental updates to a parse tree for a new version of a source file, reusing existing nodes to reduce memory and computation overhead.

The Swift parser uses recovery mechanisms to handle malformed or partial Swift code, aiming to produce as much syntax tree structure as possible. This is particularly useful for tools like formatters and editors that process incomplete input. The parser focuses on two primary error classes:

1. Unexpected Syntax: When the parser encounters a token it didn't expect, it uses a token precedence model (TokenPrecedence) to look ahead for the expected token. It attempts to skip the minimum number of tokens required to resume valid parsing.
2. Missing Syntax: When lookahead fails or is impossible, the parser synthesizes 'missing' elements to maintain tree structure. These elements have no textual content but allow the parser to continue and provide clear signals for diagnostic tools.

SwiftSyntax is designed around three core principles:

1. Immutability: The syntax tree is a persistent data structure. Once created, it cannot be changed. Modifications return a new tree that shares as much structure as possible with the original to minimize memory overhead. This makes the tree safe for concurrent use across threads without locks.
2. Source Fidelity: The library preserves every byte of the source text, including whitespace, comments, compiler directives, and the Unicode byte order mark (BOM). This is referred to as 'trivia'.
3. Resilience: The tree can represent both well-formed and ill-formed code. It handles missing syntax (represented by tokens with SourcePresence/missing) and unexpected syntax (superfluous text), allowing tools like IDEs and linters to recover gracefully during parsing.

The Swift parser uses a recursive descent approach to transform Swift source code into a syntax tree.

• Grammar Productions: Rules that define valid Swift syntax (e.g., optional-type → type '?').
• SwiftSyntax: A concrete syntax tree (CST) that faithfully represents the source text. Each grammar production corresponds to a specific syntax node type (e.g., RawOptionalTypeSyntax).
• Syntax Nodes: These nodes contain properties to access child nodes (e.g., wrappedType) and associated tokens (e.g., questionMark).
• Syntax Collections: For sequences of elements (like a list of class members), SwiftSyntax provides SyntaxCollection types.

The SwiftParser is designed to be resilient, meaning it attempts to recover from syntax errors to maintain program structure without producing side effects or failing. Errors are embedded directly in the syntax tree in two forms:

• Unexpected nodes: Syntax that does not match the Swift grammar is stored in "unexpected" child nodes that can be queried.
• Missing tokens: Required syntax that is absent in the source (e.g., a missing closing parenthesis )) is inserted into the tree as "missing" tokens. These tokens are skipped when rendering back to source text but can be used by tools to suggest or apply fixes.

Resilience in SwiftParser refers to the ability to accept ill-formed inputs and still produce a well-formed, structured output.

• Structural Well-formedness: The goal is to output as much syntactic structure as possible so clients can still interpret parts of the code.
• Recovery Mechanisms: The parser uses facilities to recover from invalid, unexpected, or missing tokens.
• Missing Nodes: The syntax tree can explicitly insert "missing" syntax elements as nodes. SwiftSyntax provides different "views" of the tree that can include these missing or unexpected nodes on demand.

If valid source code produces a syntax tree that does not correctly represent the code's semantics, use print-tree to view the tree structure. To prepare a bug report, remove as much code as possible from the file while ensuring it still produces the same invalid tree.

swift-parser-cli print-tree /path/to/file.swift