DOCSEARCHbyAlgolia
Submit Docs

SymbolicRegression.jl

repository·master·Indexed 21 days ago

https://github.com/astroautomata/symbolicregression.jl

A Julia library for discovering analytic functional forms from data by searching for symbolic mathematical expressions that optimize a given objective function. It provides a high-level MLJ interface via SRRegressor and MultitargetSRRegressor, as well as a low-level functional interface using equation_search. The library supports Pareto frontier calculation, automatic differentiation for gradients, dimensional constraints via DynamicQuantities, and integration with SymbolicUtils.jl for symbolic manipulation.

Tokens
8.2K
Snippets
29
Records
46
Agent score
64%

What's inside SymbolicRegression.jl

  1. Understand the HallOfFame concept

    master
    In SymbolicRegression.jl, the HallOfFame is the primary output object of the equation_search process. It acts as a repository that stores the best mathematical expressions discovered during the search, specifically keeping the expressions with the lowest loss encountered at each level of complexity.

  2. Define and use Template Expressions

    master

    Template expressions allow you to constrain the functional form of the search. You define a structure using the @template_spec macro, which tells the algorithm to learn specific sub-expressions (e.g., $f(x_1, x_2) + g(x_2)$).

    • The @template_spec macro defines how components combine.
    • The SRRegressor accepts an expression_spec argument.
    • Individual components of the template can be accessed from the report using get_contents(best_expr).component_name.
    • The resulting TemplateExpression can be evaluated directly: best_expr(X).
    using SymbolicRegression

# Define the structure
expression_spec = @template_spec(expressions=(f, g)) do x1, x2, x3
    f(x1, x2) + g(x2) - g(x3)
end

model = SRRegressor(
    binary_operators=(+, -, *, /),
    unary_operators=(cos,),
    niterations=500,
    maxsize=25,
    expression_spec=expression_spec,
)

# ... fit model ...

r = report(mach)
best_expr = r.equations[r.best_idx]

# Access sub-components
println("f: ", get_contents(best_expr).f)
println("g: ", get_contents(best_expr).g)

# Evaluate the whole template
best_expr(randn(3, 20))

  3. Understand Population, PopMember, and Hall of Fame

    master

    The search process manages groups of equations through several key abstractions:

    • Population: An array of equations currently being evaluated.
    • PopMember: An individual entry in a population. Each member is a tree tagged with metadata including its cost, loss, and birthdate (the iteration/time it was created).
    • Hall of Fame: A collection of the best-performing equations discovered during the search process.

  4. Use dimensional constraints in symbolic regression

    master

    You can enforce dimensional consistency by using DynamicQuantities to assign units to your input features and targets.

    Key parameters for dimensional search:

    • dimensional_constraint_penalty: A high penalty value to discourage dimensionally inconsistent expressions.
    • complexity_of_constants: Controls how constants are treated.
    • dimensionless_constants_only: Set to true to search specifically for dimensionless units.

    Expressions that satisfy the constraint will show units in the report (e.g., [m s⁻² kg]). Constants with free units are marked with [?].

    using DynamicQuantities
using SymbolicRegression

# ... setup data with units ...

function loss_fnc(prediction, target)
    scatter_loss = abs(log((abs(prediction)+1e-20) / (abs(target)+1e-20)))
    sign_loss = 10 * (sign(prediction) - sign(target))^2
    return scatter_loss + sign_loss
end

model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[square],
    elementwise_loss=loss_fnc,
    complexity_of_constants=2,
    maxsize=25,
    niterations=100,
    populations=50,
    dimensional_constraint_penalty=10^5,
)

  5. Use Parametric Expressions for optimized parameters

    master

    Parametric expressions allow you to include parameters within your equations that the search process can optimize to better fit the data.

    To control the complexity of these expressions, you can specify the maximum number of parameters allowed using the expression_options argument when initializing an SRRegressor.

  8. Implement custom mutations by subtyping AbstractMutation

    master

    You can define custom mutation logic by creating a type that subtypes AbstractMutation. To use it, you must implement the mutate! method for your type. Once implemented, pass a weighted instance of your mutation to the Options constructor using the mutations keyword argument.

    # 1. Define your type
struct MyCustomMutation <: AbstractMutation
    # ...
end

# 2. Implement the mutate! method
function mutate!(mutation::MyCustomMutation, ...)
    # ...
end

# 3. Pass to Options
Options(mutations = [MyCustomMutation() => 1.0, ...])

  10. Define and use custom loss functions

    master

    You can provide a custom loss function by passing a function that accepts either two (unweighted) or three (weighted) scalar arguments. For unweighted losses, the function should take (x, y). For weighted losses, it should take (x, y, w).

    To use a custom loss, pass it to the elementwise_loss parameter in the Options object.

    f(x, y, w) = abs(x-y)*w
options = Options(elementwise_loss=f)

  11. Run SymbolicRegression.jl in Interactive Mode via salloc

    master

    For prototyping, you can run SymbolicRegression.jl interactively using salloc.

    1. Request resources:
      salloc -p YOUR_PARTITION -N 2
    2. Identify your allocated node using squeue -u $USER and connect via ssh.
    3. Launch your script while manually declaring the number of tasks via the SLURM_NTASKS environment variable. It is also recommended to set JULIA_NUM_THREADS=1 to avoid overusing CPU resources.
    SLURM_NTASKS=127 julia --project=. script.jl

  12. Create custom expression types by extending AbstractExpression

    master

    To define new functional forms or expressions with extra parameters, create a type that extends AbstractExpression.

    1. Implement the necessary methods (refer to DynamicExpressions.jl for the full interface requirements).
    2. Use ExpressionInterface (based on Interfaces.jl) to verify your implementation.
    3. Pass your type to the Options constructor via the expression_type keyword.
    4. Use expression_options (as a NamedTuple) for any additional configuration required by your expression.
    5. If your expression requires extra initialization parameters, you may need to overload SymbolicRegression.ExpressionBuilder.extra_init_params.
    # Example pattern for custom expressions
Options(
    expression_type = MyCustomExpressionType,
    expression_options = (param1 = value1, param2 = value2)
)