phplrt

To avoid production errors and large dependency footprints, phplrt distinguishes between the Runtime and Build-time describers.

The Correct Workflow

1. Develop: Use .pp3 grammar files or in-PHP LexerBuilder/ParserBuilder to define your grammar.
2. Generate: Use the phplrt/compiler (CLI or PHP API) to compile your grammar into a standalone PHP class.
3. Commit: Commit the generated parser class to your repository.
4. Deploy: In production, depend only on phplrt/runtime. The generated class is a plain PHP class that extends \Phplrt\Parser\Parser and requires no compiler or builders to run.

The Anti-Pattern to Avoid

Do not call (new Compiler())->...->getParser() or $builder->build()->toParser() at request time in production. This forces heavy dev-dependencies (like symfony/console or twig) into your production environment and recompiles the grammar on every request.

# 1. Install production runtime
composer require phplrt/runtime

# 2. Install dev compiler
composer require --dev phplrt/compiler

# 3. Generate the parser (Dev step)
vendor/bin/phplrt compile grammar.pp3 src/MyParser.php \
    --class=MyParser --namespace='App\Parser'

# 4. Use the generated parser in production (No compiler needed)
$parser = new App\Parser\MyParser();
$ast = $parser->parse(Phplrt\Source\StringSource::createFromString($input));

The second argument of a reducer function changes based on the rule type. Getting this wrong is a common source of errors.

Reducer Argument Shapes

• Concatenation / Repetition: Receives a flat array of the children's reduced values. If you reference a token but don't use a reducer for it, its slot in the array will contain the raw TokenInterface.
• Alternation / Optional / Terminal: Receives a single value. For an absent optional, the value is an empty array [] (not null).
• Kept Terminal (addTokenReference): If no reducer is provided, the reducer receives the raw TokenInterface. Access data via $token->value, $token->name, or $token->children.

Rule of thumb: Sequence $\rightarrow$ array, Choice/Terminal $\rightarrow$ scalar.

The Flattening Trap

If a rule (Rule B) is a Concatenation/Repetition and returns a bare array from its reducer, that array will be flattened into the parent rule's (Rule A) array. This makes it impossible for Rule A to distinguish where Rule B's children end.

To prevent flattening: Ensure the nested rule's reducer returns something that is not a bare array, such as an object or a scalar.

// BAD: $pair's array merges into $line's array
$pair = $builder->addConcatenation([$word, $word])
    ->setReducer(static fn($ctx, $kids) => $kids);

$line = $builder->addConcatenation([$pair, $word])
    ->setReducer(static fn($ctx, $kids) => $kids);
// $kids is ['a', 'b', 'c'], NOT [['a', 'b'], 'c']

// GOOD: $pair returns an object, so it survives intact
$pair = $builder->addConcatenation([$word, $word])
    ->setReducer(static fn($ctx, $kids) => (object) ['pair' => $kids]);

Phplrt uses a grammar syntax (e.g., .pp3 files) to define how text is recognized.

Key concepts:

• %skip: Defines patterns (like whitespace) to be ignored.
• %token: Defines terminal symbols (tokens) using regular expressions.
• %pragma root: Specifies the starting rule for recognition.
• Rules: Define how tokens and other rules combine.
  • <T_TOKEN>: Reads a token and keeps its value.
  • ::T_TOKEN::: Reads a token but discards it.
  • *: Quantifier meaning "zero or more times".
• Reducers: PHP code blocks within a rule (using ->) that execute when a rule matches, transforming the matched components into a specific value.

// grammar.pp3

%skip  T_WHITESPACE  \s++

%token T_NUMBER  \d++
%token T_PLUS    \+

// Recognition starts from this rule
%pragma root Sum

Sum -> { return \is_array($children) ? \array_sum($children) : $children; }
  : Number() (::T_PLUS:: Number())*
  ;

Number -> { return (int) $children->value; }
  : <T_NUMBER>
  ;

Phplrt supports nested lexing through a mechanism where an opening token triggers a switch to a sub-lexer. This allows you to define a sub-language (like YAML inside Markdown frontmatter or code inside a fenced block) without cluttering the main grammar.

The Mechanism

1. Define the Parent Pattern: Use ->enter('lexer_name') on a pattern definition to signal that the token found should switch the active lexer to a named sub-lexer.
2. Define the Sub-Lexer: Use addLexer('lexer_name') to create a regex-based sub-lexer. Inside this sub-lexer, use ->exit() on a pattern to return control to the parent lexer.
3. Accessing Data: The opening token becomes a TokenEmbedding. Instead of looking for inner token names in the main parser, the parser references the opening token's TokenDefinition. The sub-tokens are stored in the $token->children array. Each child has a ->name and a ->value.

Key Behaviors

• Line Anchors: Inside a sub-lexer, ^ matches the start of each line (multiline mode is default). In the parent lexer, \A matches only the start of the file.
• Encapsulation: The outer parser only needs to know about the opening token. The sub-format logic is contained within the sub-lexer and the reducer.

$lexer = new LexerBuilder();

// Opening token ENTERS a sub-lexer.
$fm = $lexer->addPattern('\A---+[^\n]*\n?', 'T_FM_DELIM')->enter('frontmatter');

$sub = $lexer->addLexer('frontmatter');
$sub->addPattern('^---+[^\n]*\n?', 'T_FM_DELIM_CLOSE')->exit(); // EXIT back to parent
$sub->addPattern('^[A-Za-z_][\w-]*:', 'T_FM_KEY');
$sub->addPattern('[^\n]*\n?', 'T_FM_VALUE');

When using nested lexers, the parent token contains all the tokens matched by the sub-lexer in its children property. You should iterate through these children in your reducer to reconstruct the nested data.

To process the embedding, use $token->children and check the ->name of each child. If a sub-lexer contains hidden tokens (like comments), you can skip them by checking their name.

$frontmatter = $builder->addTokenReference($fm)
    ->setReducer(static function ($ctx, $token) use ($inline) {
        $attrs = [];
        $key = null;
        foreach ($token->children as $child) {
            switch ($child->name) {
                case 'T_FM_KEY':
                    $key = rtrim($child->value, ':');
                    $attrs[$key] = '';
                    break;
                case 'T_FM_VALUE':
                    if ($key !== null) { $attrs[$key] .= trim($child->value); }
                    break;
                case 'T_FM_DELIM_CLOSE':
                    break 2; // stop at the closing line
            }
        }
        return new Node('frontmatter', $attrs);
    });