PHP-Parser

repository·master·Indexed 12 days ago

https://github.com/nikic/php-parser

A PHP-based library for parsing PHP code into an Abstract Syntax Tree (AST) to simplify static code analysis and manipulation. It includes a Lexer, Parser, Name Resolver, and Pretty Printer, as well as fluent AST builders via BuilderFactory. Supports PHP 7 and 8, with limited support for PHP 5, and provides a php-parse CLI tool for transforming source code into various AST output formats.

Tokens
26.3K
Snippets
67
Records
124
Agent score
94%

What's inside PHP-Parser

  1. Explore additional features of PHP Parser

    master

    Beyond basic parsing, this package provides several tools for working with the AST:

    • Pretty Printing: Converts an AST back into PHP code.
    • Serialization: Supports serializing and unserializing the node tree to JSON.
    • Dumping: Provides human-readable dumping of the node tree for debugging.
    • AST Manipulation: Includes infrastructure for traversing and modifying the tree via node traversers and node visitors.
    • Name Resolution: Includes a node visitor specifically for resolving namespaced names.
  2. Get started with PHP Parser

    master
    PHP Parser is a tool for parsing PHP code into an Abstract Syntax Tree (AST). You can use it to analyze, transform, or generate PHP code. The library provides a complete set of tools including a Lexer, a Parser, a Name Resolver, and a Pretty Printer to work with the AST.
  3. How NameResolver handles unqualified names

    master
    In version 3.0, the NameResolver now resolves unqualified function and constant names in the global namespace into fully qualified names. For example, a call to foo() in the global namespace will resolve to \foo(). If static resolution is not possible, a namespacedName attribute is added to the node containing the namespaced variant.
  4. What the PHP Parser produces and supports

    master

    The parser converts PHP source code into an Abstract Syntax Tree (AST), also known as a node tree.

    Supported Versions

    • PHP 7 and PHP 8: Full support.
    • PHP 5: PHP-Parser 5.x has limited support. Some variable expressions (like $$foo[0]) will always be interpreted using PHP 7 logic, and certain declarations like global $$var[0] will cause parse errors (though error recovery mode can mitigate this).
    • Cross-version parsing: The package includes a wrapper to emulate tokens from newer PHP versions, allowing you to parse newer code (e.g., PHP 8.4) even when running on an older PHP version (e.g., 7.4).

    Output Structure

    An AST represents the logical structure of the code. For example, the code <?php echo 'Hi', 'World'; produces a tree structure similar to this:

    array(
        0: Stmt_Echo(
            exprs: array(
                0: Scalar_String(
                    value: Hi
                )
                1: Scalar_String(
                    value: World
                )
            )
        )
    )
  5. Understand the difference between an AST and a token stream

    master

    When processing PHP source code, you have two primary approaches:

    1. Abstract Syntax Tree (AST): Produced by this parser. It provides a high-level, robust representation of the code structure. It abstracts away syntactic variations (e.g., different ways to write variable names like $foo vs ${'foobar'}) so you can focus on logic rather than syntax. This is ideal for static analysis and code manipulation.
    2. Token Stream: Generated by PHP's native token_get_all. This is a low-level stream of tokens. It is useful for analyzing exact file formatting but is significantly harder to use for complex structural analysis because you must manually handle all syntactic permutations.

    Note that the AST produced by this parser does not contain whitespace information, though it retains accurate position information for inspecting precise formatting and preserves most comments.

  6. Handle Identifier changes in node subnodes (v4.0)

    master

    In version 4.0, many subnodes that previously held simple strings now store Identifier nodes (or VarLikeIdentifier nodes if they have the form $ident).

    While constructors automatically convert strings to Identifiers and Identifiers implement __toString(), you must update any code that uses:

    • is_string() checks on these subnodes.
    • Type-strict comparisons.
    • Strict-mode operations.

    Affected subnodes include:

    • Const_::$name
    • NullableType::$type (simple types)
    • Param::$type (simple types)
    • Expr\ClassConstFetch::$name
    • Expr\Closure::$returnType (simple types)
    • Expr\MethodCall::$name
    • Expr\PropertyFetch::$name
    • Expr\StaticCall::$name
    • Expr\StaticPropertyFetch::$name (uses VarLikeIdentifier)
    • Stmt\Class_::$name
    • Stmt\ClassMethod::$name
    • Stmt\ClassMethod::$returnType (simple types)
    • Stmt\Function_::$name
    • Stmt\Function_::$returnType (simple types)
    • Stmt\Goto_::$name
    • Stmt\Interface_::$name
    • Stmt\Label::$name
    • Stmt\PropertyProperty::$name (uses VarLikeIdentifier)
    • Stmt\TraitUseAdaptation\Alias::$method
    • Stmt\TraitUseAdaptation\Alias::$newName
    • Stmt\TraitUseAdaptation\Precedence::$method
    • Stmt\Trait_::$name
    • Stmt\UseUse::$alias
  7. Avoid cyclic reference issues in the AST using WeakReferences

    master

    Using NodeConnectingVisitor or ParentConnectingVisitor introduces cyclic references (e.g., a child pointing to a parent that points back to the child). This can prevent immediate garbage collection and lead to memory or performance issues.

    To mitigate this, initialize NodeConnectingVisitor with the $weakReferences parameter set to true. This wraps the added attributes in WeakReference objects.

    When using weak references, the attribute keys change to:

    • weak_parent
    • weak_previous
    • weak_next
    // Use true to enable WeakReferences to prevent memory/performance issues from cycles
    $traverser = new NodeTraverser(new NodeConnectingVisitor(true));
    
    // ... traverse ...
    
    // Access via weak attribute keys:
    // $parent = $node->getAttribute('weak_parent');
    // $prev   = $node->getAttribute('weak_previous');
    // $next   = $node->getAttribute('weak_next');
  8. How the Lexer works with the Parser

    master

    The lexer is responsible for providing tokens to the parser. In most use cases, you do not need to interact with the lexer directly. Instead, you use PhpParser\ParserFactory to create an appropriate parser, which handles lexing internally. After parsing, you can retrieve the tokens produced by the lexer using PhpParser\Parser::getTokens().

    // Typical workflow
    $parser = (new PhpParser\ParserFactory())->createForHostVersion();
    $stmts = $parser->parse($code);
    $tokens = $parser->getTokens(); // Retrieve tokens used for the last parse
  9. Understand changes to Node::getType() output

    master

    The Node::getType() method returns names using underscores instead of namespace separators and does not include trailing underscores used in class names.

    In version 1.0, several node classes were renamed or moved to different namespaces, which changes the string returned by Node::getType(). If your code compares getType() results to specific strings or uses them in custom pretty printers, you must update your logic to match the new mappings.

    Key changes include:

    • Assignment operators moved from Expr_Assign... to Expr_AssignOp_... (e.g., Expr_AssignPlus becomes Expr_AssignOp_Plus).
    • Binary operators moved from Expr_... to Expr_BinaryOp_... (e.g., Expr_Plus becomes Expr_BinaryOp_Plus).
    • Magic constants moved from Scalar_...Const to Scalar_MagicConst_... (e.g., Scalar_ClassConst becomes Scalar_MagicConst_Class).
    // Example of mapping changes:
    Expr_AssignPlus       => Expr_AssignOp_Plus
    Expr_Plus              => Expr_BinaryOp_Plus
    Scalar_ClassConst      => Scalar_MagicConst_Class
  10. How formatting-preserving pretty printing works

    master

    For automated refactoring where you want to modify only specific parts of the code while leaving the rest of the formatting untouched, use the printFormatPreserving method. This mode attempts to preserve the formatting of AST nodes that have not changed.

    To use this functionality, you must follow a specific workflow:

    1. Parse the original code to get statements and tokens.
    2. Use a CloningVisitor to create a copy of the AST before making any modifications.
    3. Modify the cloned AST.
    4. Call printFormatPreserving using the new statements, the original statements, and the original tokens.

    Warning: If using name resolution, you should likely disable the replaceNodes option to prevent resolved names from being written directly into the AST, which can cause unintended changes in the output.

    use PhpParser\{NodeTraverser, NodeVisitor, ParserFactory, PrettyPrinter};
    
    $parser = (new ParserFactory())->createForHostVersion();
    $oldStmts = $parser->parse($code);
    $oldTokens = $parser->getTokens();
    
    // Run CloningVisitor before making changes to the AST.
    $traverser = new NodeTraverser(new NodeVisitor\CloningVisitor());
    $newStmts = $traverser->traverse($oldStmts);
    
    // MODIFY $newStmts HERE
    
    $printer = new PrettyPrinter\Standard();
    $newCode = $printer->printFormatPreserving($newStmts, $oldStmts, $oldTokens);
  11. Short-circuit AST traversal

    master

    To improve performance, you can instruct the NodeTraverser to skip parts of the tree or stop entirely by returning specific constants from enterNode():

    • NodeVisitor::DONT_TRAVERSE_CHILDREN: Skips the children of the current node. Useful when you know no nested structures of interest exist (e.g., skipping children of a Class_ node if you only want top-level classes).
    • NodeVisitor::STOP_TRAVERSAL: Aborts the entire traversal immediately. Useful when you only need to find a single specific node.
    public function enterNode(Node $node) {
        if ($node instanceof Node\Stmt\Class_) {
            // Found a class, don't bother looking at its contents
            return NodeVisitor::DONT_TRAVERSE_CHILDREN;
        }
        if ($node instanceof Node\Stmt\Class_ && $node->name === 'TargetClass') {
            // Found exactly what we wanted, stop everything
            return NodeVisitor::STOP_TRAVERSAL;
        }
    }
  12. Implement a NodeVisitor

    master

    A NodeVisitor defines how to react to nodes during traversal. While you can implement the NodeVisitor interface directly, it is recommended to extend NodeVisitorAbstract to avoid implementing all methods.

    Key methods:

    • beforeTraverse(array $nodes): Called once before the entire traversal starts. Use for setup.
    • enterNode(Node $node): Called when a node is first encountered (preorder), before its children are visited.
    • leaveNode(Node $node): Called after all children of a node have been visited (postorder). Use this to perform modifications based on collected information from children.
    • afterTraverse(array $nodes): Called once after the entire traversal finishes. Use for cleanup.