Kusto Query Language (KQL) Implementation

You can extend the parser's knowledge of functions and aggregates (even if they don't exist on the server) by modifying the GlobalState before parsing. This is useful for simulating specific environments or adding custom logic.

• Use WithFunctions to add FunctionSymbol instances.
• Use WithAggregates to add aggregate functions.

Note that removing functions or aggregates from the GlobalState will cause the parser to produce errors when they are encountered in a query.

// Adding a fake function
var fnFake = new FunctionSymbol("fake", ScalarTypes.Real, new Parameter("x", ScalarTypes.Long), new Parameter("y", ScalarTypes.Long));
var globals = GlobalState.Default.WithFunctions(globals.Functions.Concat(new [] {fnFake}).ToArray());
var code = KustoCode.ParseAndAnalyze("print fake(10)", globals);

// Adding a fake aggregate
var fnMinMax = new FunctionSymbol("minmax", ScalarTypes.Real, new Parameter("x", ScalarTypes.Real));
var globals = GlobalState.Default.WithAggregates(globals.Aggregates.Concat(new [] {fnMinMax}).ToArray());
var code = KustoCode.ParseAndAnalyze("T | summarize minmax(c)", globals);

To identify which tables are used in a Kusto query, you can walk the syntax tree of a KustoCode object. A basic approach checks for TableSymbol references or expressions where the ResultType is a TableSymbol.

Note: A basic walk may miss tables referenced inside the bodies of called functions. To capture these, you must use a recursive technique that calls GetCalledFunctionBody() on nodes to examine the function's internal syntax.

For a production-ready implementation, it is recommended to use the Kusto.Toolkit NuGet package.

    public static HashSet<TableSymbol> GetDatabaseTables(KustoCode code)
    {
        var tables = new HashSet<TableSymbol>();
        GatherTables(code.Syntax);
        return tables;

        void GatherTables(SyntaxNode root)
        {
            SyntaxElement.WalkNodes(root,
                fnBefore: n =>
                {
                    if (n.ReferencedSymbol is TableSymbol t
                        && code.Globals.IsDatabaseTable(t))
                    {
                        tables.Add(t);
                    }
                    else if (n is Expression e
                        && e.ResultType is TableSymbol ts
                        && code.Globals.IsDatabaseTable(ts))
                    {
                        tables.Add(ts);
                    }
                    else if (n.GetCalledFunctionBody() is SyntaxNode body)
                    {
                        GatherTables(body);
                    }
                },
                fnDescend: n =>
                    !(n is FunctionDeclaration)
                );
        }
    }

To enable semantic analysis for specific tables and functions, you must declare them in the GlobalState before calling ParseAndAnalyze.

1. Define Symbols

• Tables: Create a TableSymbol with a name and a schema string.
• Functions: Create a FunctionSymbol with a name and a body. Functions can have parameters or be parameterless.
• Databases: Create a DatabaseSymbol by passing the name and the collection of tables and functions.

2. Register in GlobalState

Since GlobalState is immutable, use the With methods to add your definitions. You can add a database to the default scope or add a ClusterSymbol containing multiple databases.

Accessing non-default entities:

• If a database is not the default, use the database() function in your query.
• If a cluster is not the default, use the cluster() function in your query.

// 1. Declare symbols
var shapes = new TableSymbol("Shapes", "(id: string, width: real, height: real)");
var tallshapes = new FunctionSymbol("TallShapes", "{ Shapes | width < height; }");
var shortshapes = new FunctionSymbol("ShortShapes", "(maxHeight: real)", "{ Shapes | height < maxHeight; }");

// 2. Create database
var mydb = new DatabaseSymbol("mydb", shapes, tallshapes, shortshapes);

// 3. Add to GlobalState
var globalsWithMyDb = GlobalState.Default.WithDatabase(mydb);

// Alternatively, add via a cluster
var mycluster = new ClusterSymbol("mycluster.kusto.windows.net", mydb);
var globalsWithMyClusterAdded = GlobalState.Globals.AddOrReplaceCluster(mycluster);

To identify which columns in a query originate from the provided database schema, you must traverse the syntax tree and check the ReferencedSymbol of nodes.

Because columns can be renamed or introduced via operators (like union), a ColumnSymbol might not be directly found in the GlobalState. In such cases, you should check the OriginalColumns property of the ColumnSymbol to find the underlying schema columns.

Additionally, if a query calls a database function, the columns referenced inside that function's body will not appear in the main query's syntax tree. To find these, use n.GetCalledFunctionBody() to retrieve the syntax tree of the function and traverse it recursively.

public static HashSet<ColumnSymbol> GetDatabaseTableColumns(KustoCode code)
{
    var columns = new HashSet<ColumnSymbol>();
    GatherColumns(code.Syntax);
    return columns;

    void GatherColumns(SyntaxNode root)
    {
        SyntaxElement.WalkNodes(root,
            fnBefore: n =>
            {
                if (n.ReferencedSymbol is ColumnSymbol c)
                {
                    AddDatabaseTableColumns(c, code.Globals, columns);
                }
                else if (n.GetCalledFunctionBody() is SyntaxNode body)
                {
                    GatherColumns(body);
                }
            },
            fnDescend: n =>
                // skip descending into function declarations since their bodies will be examined by the code above
                !(n is FunctionDeclaration)
            );
    }
}

To distinguish between different entities with the same name (like two different columns named a) and to check for semantic errors, use KustoCode.ParseAndAnalyze.

This method requires a GlobalState instance, which acts as a provider for symbols (definitions of database tables, functions, etc.). When semantic analysis is performed, syntax nodes gain ReferencedSymbol and ResultType properties, allowing you to identify exactly which column, variable, or table is being referenced.

var globals = GlobalState.Default.WithDatabase(
    new DatabaseSymbol("db",
        new TableSymbol("T", "(a: real, b: real)")));

var query = "T | project a = a + b | where a > 10.0";
var code = KustoCode.ParseAndAnalyze(query, globals);

// Accessing the specific column symbol via the syntax tree
var columnA = globals.Database.Tables.First(t => t.Name == "db").GetColumn("a");
var referencesToA = code.Syntax.GetDescendants<NameReference>(n => n.ReferencedSymbol == columnA);