Loretta Toolkit
repository·main·Indexed 41 days ago
https://github.com/lorettadevs/lorettaA Roslyn-based C# and (G)Lua toolkit for lexing, parsing, code analysis, transformation, and code generation. It supports various Lua dialects including Standard Lua (5.1-5.4), LuaJIT (2.0, 2.1), Luau (Roblox), Garry's Mod (GLua), and FiveM. Available via NuGet as Loretta.CodeAnalysis.Lua and Loretta.CodeAnalysis.Lua.Experimental, the latter providing experimental features like constant folding and a Lua minifier.
What's inside Loretta
- Loretta is a C# (G)Lua toolkit designed for lexing, parsing, code analysis, transformation, and code generation. It is built from scratch based on Roslyn and the official Lua 5.4 syntax, providing support for multiple Lua dialects and extensions.
What is SourceText?
mainSourceTextis a core abstraction inLoretta.CodeAnalysis.Textdesigned for efficient code representation.Key Responsibilities:
- Memory Efficiency: Stores code in a format that avoids the Large Object Heap (LOH).
- Text Manipulation: Allows obtaining specific characters, substrings, and splitting code into
TextLines. - Mapping: Maps a
TextSpanto aLinePositionSpanfor accurate error reporting. - Versioning/Diffing: Provides methods to obtain file checksums, calculate changes between versions via
SourceText.GetTextChanges, or apply changes viaSourceText.WithChanges.
Use LuaSyntaxRewriter to modify syntax trees
mainThe
LuaSyntaxRewriteris a fundamental building block for transforming syntax trees. It allows you to replace specific nodes in a tree without manually managing parent-child relationships. You implement a custom rewriter by inheriting fromLuaSyntaxRewriterand overridingVisitmethods for the specific node types you want to transform (e.g.,VisitCompilationUnitorVisitFunctionCallExpression).When a node is modified, you typically return the new node using methods like
node.Update(...)or by returning a newly constructed node viaSyntaxFactory.class MyRewriter : LuaSyntaxRewriter { public override SyntaxNode? VisitSomeNode(SomeNodeSyntax node) { // Transform the node and return it return node.Update(...); } }Understand Trivia in Loretta
mainTrivia refers to extraneous syntax that does not impact parsing, such as whitespaces, line breaks, comments, and shebangs (
#!). Trivia is attached to the preceding or following token:- Leading trivia: All trivia located from the first line break after the previous token up to the current token.
- Trailing trivia: All trivia after a token up to (and including) the first line break.
You can manipulate trivia using
GetTrailingTrivia()andWithTrailingTrivia(SyntaxTriviaList).How LuaSyntaxWalker works for tree traversal
mainA
LuaSyntaxWalkeris a fundamental building block used to traverse a syntax tree recursively. By inheriting fromLuaSyntaxWalker, you can override specificVisit...methods to perform actions when the walker encounters certain node types (e.g.,VisitFunctionCallExpression).Key implementation details:
- Depth Control: When calling the base constructor, you can specify
SyntaxWalkerDepth.Nodeto limit traversal to nodes, ignoring everything below them. - Recursive Traversal: When overriding a
Visitmethod, you must callbase.Visit[MethodName](node)if you want the walker to continue traversing the children of that node. Failing to do so will stop the traversal for that branch.
internal class MyWalker : LuaSyntaxWalker { public MyWalker() : base(SyntaxWalkerDepth.Node) { } public override void VisitFunctionCallExpression(FunctionCallExpressionSyntax node) { // Do something with the node base.VisitFunctionCallExpression(node); // Crucial for continued traversal } }- Depth Control: When calling the base constructor, you can specify
Access Lua scoping and variable information
mainTo perform semantic analysis like retrieving scopes or variables, you must first create a
Scriptinstance from yourLuaSyntaxTrees.Once a
Scriptis created, you can:- Get the global scope via
Script.RootScope. - Retrieve a specific scope using
Script.GetScope(SyntaxNode)orScript.FindScope(SyntaxNode, ScopeKind). - Retrieve variable information using
Script.GetVariable(SyntaxNode). - Retrieve labels for
gotostatements usingScript.GetLabel(SyntaxNode)(works onGotoStatementSyntaxorGotoLabelStatementSyntax).
- Get the global scope via
Supported Lua dialects and syntax extensions in Loretta
mainLoretta provides broad compatibility across several Lua environments and extensions:
Core Lua Versions
- Lua 5.1, 5.2, 5.3, and 5.4
- LuaJIT 2.0 and 2.1
Dialect Extensions
- Garry's Mod Lua (glua): Supports C-style comments (
//and/* */) and logical operators (&&,||,!=,!). - Luau (Roblox Lua): Partial support for compound assignments (
+=,-=,*=,/=,^=,%=,..=), if expressions (if a then b else c), and typed Lua syntax. - FiveM: Supports parsing of hash string syntax.
- LuaJIT Compatibility: Accepts identifiers containing characters above
127/0x7F(e.g., emojis).
continueKeyword SupportLoretta allows configuring how the
continuekeyword is handled via different options:- No
continuesupport. - Roblox style:
continueis treated as a contextual keyword. - Garry's Mod style:
continueis treated as a full-fledged keyword.
Supported Lua dialects and syntax extensions
mainLoretta is a toolkit for lexing, parsing, and analyzing Lua-based code. It supports a wide range of dialects including:
- Standard Lua: 5.1, 5.2, 5.3, and 5.4.
- LuaJIT: 2.0 and 2.1 (including LuaJIT's specific identifier support for characters above
0x7F). - Luau (Roblox): Supports compound assignments (
+=,-=, etc.), if expressions, and typed Lua syntax. - Garry's Mod (GLua): Supports C-style comments (
//and/* */) and specific operators (&&,||,!=,!). - FiveM: Supports FiveM's hash string syntax.
Continue Keyword Support: You can configure how the
continuekeyword is handled via options:- No
continuesupport. - Luau style:
continueis treated as a contextual keyword. - Garry's Mod style:
continueis treated as a full-fledged keyword.
Check if a Lua node can be converted into a local variable
mainA Lua syntax node is considered eligible to be turned into a local variable if it is either:
- An
IdentifierNameSyntax(e.g.,print). - A
MemberAccessExpressionSyntaxwhose base expression is also eligible (e.g.,math.ceilora.b.c).
Complex expressions like
(1 + 1).aare not eligible.You can implement this check using the
IsKindextension method fromLoretta.CodeAnalysis.LuaExtensionsto verify theSyntaxKindof the node.static bool canTurnIntoLocal(SyntaxNode node) { if (node.IsKind(SyntaxKind.IdentifierName)) return true; else if (node.IsKind(SyntaxKind.MemberAccessExpression)) return canTurnIntoLocal(((MemberAccessExpressionSyntax) node).Expression; else return false; }- An
Create Lua nodes with SyntaxFactory
mainThe
SyntaxFactoryis a static class used to instantiate all types of Lua syntax nodes, including identifiers, statements, and lists. Because Loretta syntax trees are immutable, all operations that modify a node return a new instance.Common factory methods include:
SyntaxFactory.IdentifierName(string text): Creates an identifier.SyntaxFactory.LocalVariableDeclarationStatement(...): Creates a local variable declaration.SyntaxFactory.SeparatedList(IEnumerable<T> items): Creates a syntax list where items are separated by specific tokens.
How the Lua Minifier works
mainThe minifier operates in two main stages:
1. Renaming
The renamer iterates through the script and manages variable names using slots:
- New Variables: When a new variable is encountered, the minifier requests a slot from the
ISlotAllocatorand converts that slot into a name using theNamingStrategy. - Existing Variables: When an existing variable is encountered, it is renamed using the strategy. If it is the last use of that variable, the minifier releases the slot back to the allocator.
2. Trivia Removal
This step removes all 'trivia' (comments and whitespace) from the tree, inserting spaces only where they are syntactically necessary.
- New Variables: When a new variable is encountered, the minifier requests a slot from the
Use SegmentedArray<T> to avoid Large Object Heap allocations
mainThe
SegmentedArray<T>type is a specialized collection designed as a high-performance alternative toT[]. It provides O(1) reads and writes and maintains storage density/data locality similar to a standard array, but segments the underlying storage to prevent the underlying arrays from being allocated in the Large Object Heap (LOH).When to use
SegmentedArray<T>:- You are currently using
T[]and need to maintain its performance characteristics (O(1) access). - You do not need to call
.AsMemory()or.AsSpan()on the instance, as these operations require the entire collection to be continuous in memory. - Application profiling indicates that
T[]allocations in the Large Object Heap are causing performance issues.
- You are currently using