repository·master·Indexed 40 days ago
https://github.com/justindujardin/mathyA framework for solving mathematical problems step-by-step by integrating machine learning and planning algorithms. It includes mathy_core for expression parsing and transformation via BaseRule, and Mathy Envs for Reinforcement Learning (RL) environments. The system provides a CLI for simplifying polynomial expressions and generating problem sets.
Mathy is a mathematical toolset with several core capabilities:
The Fractal Monte Carlo (FMC) algorithm is provided by the mathy.fragile module. It uses a swarm of walkers to explore environments and find optimal paths to solutions, making it useful for navigating large, sparse action spaces without full model training.
Key components for an FMC implementation include:
mathy.fragile.env.DiscreteEnv: The base class for discrete environments.mathy.fragile.models.DiscreteModel: The base class for action selection models.mathy.fragile.states.StatesEnv, StatesModel, and StatesWalkers: Classes managing environment, model, and walker states.mathy.fragile.swarm.Swarm: The main orchestrator for the swarm planning process.mathy.fragile.distributed_env.ParallelEnv: A wrapper to enable multiprocessing for the swarm.The ActorCritic module implements the dual-structure learning mechanism essential for PPO:
Softmax layer. It supports optional action masking via the use_masked_actions flag, which is useful when the action space is sparse or contains many invalid actions.Key methods:
act(state): Returns the sampled action, its action_logprob, and the state_val (critic's estimate).evaluate(state, action): Used during the update phase to calculate action_logprobs, state_values, and dist_entropy for the current policy.import torch.nn as nn
from torch.distributions import Categorical
class ActorCritic(nn.Module):
def __init__(self, state_dim, action_dim):
# ... initialization of actor and critic networks ...
def act(self, state):
# ... returns action.detach(), action_logprob.detach(), state_val.detach() ...
def evaluate(self, state, action):
# ... returns action_logprobs, state_values, dist_entropy ...Mathy environments are reinforcement learning environments designed for manipulating math trees using a rules system. They follow these core mechanics:
Environments are implemented via a base interface and wrapped in classes compatible with gym or gymnasium libraries.
The RolloutBuffer is a storage mechanism used during a single policy iteration in Proximal Policy Optimization (PPO). It collects trajectories generated by the agent's interaction with the environment, specifically storing:
actions: The actions taken.states: The states encountered.logprobs: The log probabilities of the actions under the current policy.rewards: The rewards received.state_values: The estimated values of the states.is_terminals: Boolean flags indicating if a state is terminal.Once the policy is updated using the data in the buffer, the buffer must be cleared using the .clear() method to prepare for the next iteration.
class RolloutBuffer:
def __init__(self):
self.actions = []
self.states = []
self.logprobs = []
self.rewards = []
self.state_values = []
self.is_terminals = []
def clear(self):
del self.actions[:]
del self.states[:]
del self.logprobs[:]
del self.rewards[:]
del self.state_values[:]
del self.is_terminals[:]Mathy can solve problems using a swarm-planning algorithm from the fragile library. This algorithm does not require a pre-trained model; instead, it simulates many different possible actions simultaneously and selects the most rewarding one to execute. This approach can be applied to any task exposed by Mathy to achieve decent results without the need for training.
To use this, you can implement a swarm-planning loop that interacts with a Mathy environment by sampling actions and evaluating their rewards.
# Note: The specific implementation details are contained in the snippet
# ./snippets/examples/swarm_random_task.pyTo follow the examples in this notebook, you need to install thinc and mathy_core via pip.
!pip install "thinc>=8.0.0a0" mathy_coreTo maximize efficiency in FMC, implement a batch-capable environment wrapper. This wrapper should implement a step_batch method to allow the swarm to interact with multiple environment instances simultaneously, rather than stepping them one by one.
An implementation typically involves:
PlanningEnvironment that wraps a standard gym environment and supports state-based stepping (set_state and get_state).step_batch method that iterates through actions and states to return arrays of new_states, observs, rewards, terminals, and infos.FMCEnvironment (inheriting from DiscreteEnv) that uses the PlanningEnvironment and implements make_transitions to map the batch results to the format expected by mathy.fragile.You can programmatically identify and count 'like terms' in a math expression by parsing it with ExpressionParser and using get_terms and get_term_ex from mathy_core.
Workflow:
MathExpression using ExpressionParser().parse(text).get_terms(expression).get_term_ex(term_node) to extract the variable and exponent.mathy_term_string) to identify which are 'like'.Example logic:
from mathy_core import MathExpression, ExpressionParser, get_terms, get_term_ex
from mathy_core.problems import mathy_term_string
parser = ExpressionParser()
expression = parser.parse("4x + 2x - x + 7")
term_nodes = get_terms(expression)
# Grouping logic example
for term_node in term_nodes:
ex = get_term_ex(term_node)
key = mathy_term_string(variable=ex.variable, exponent=ex.exponent)
# Use 'key' to group terms in a dictionaryfrom mathy_core import MathExpression, ExpressionParser, get_terms, get_term_ex
from mathy_core.problems import mathy_term_string
parser = ExpressionParser()
expression = parser.parse("4x + 2x - x + 7")
term_nodes = get_terms(expression)
for term_node in term_nodes:
ex = get_term_ex(term_node)
key = mathy_term_string(variable=ex.variable, exponent=ex.exponent)You can install the Mathy Python package directly from the local file system using pip. Using the -e flag allows you to make changes to the source code and have them reflected immediately without re-installing.
pip install -e ./mathyTo execute all tests and generate a visual coverage report, run the test.sh script from the root folder. The results and coverage data will be stored in the ./htmlcov/ directory. You can view the interactive coverage report by opening ./htmlcov/index.html in a web browser.
sh tools/test.sh