MetaMath Documentation

To use the data generation scripts, you must provide an API key. Set the APIKEY environment variable in your shell configuration file (e.g., ~/.bash_profile) to ensure it is available during script execution.

export APIKEY="YOUR_API_KEY"

To evaluate the reversal mathematical reasoning ability of your models, use the GSM8K_Backward dataset released on Hugging Face.

https://huggingface.co/datasets/meta-math/GSM8K_Backward

The MetaMath data generation process is divided into several stages. Each stage is executed by navigating to the code directory and running a specific shell script.

Pipeline Stages:

1. Create backward questions: Generates backward questions.
2. AnsAug: Performs Answer Augmentation.
3. Rephrasing: Performs question rephrasing.
4. Self-Verification: Performs self-verification.
5. FOBAR: Runs the FOBAR procedure (backward generation).

# 0. Create backward questions
cd code
bash -x run_create_backward_questions.sh

# 1. AnsAug
cd code
bash -x run_forward.sh

# 2. Rephrasing
cd code
bash -x run_rephrase.sh

# 3. Self-Verification
cd code
bash -x run_sv.sh

# 4. FOBAR
cd code
bash -x run_backward.sh

To set up MetaMath, clone the repository and install the required Python dependencies using pip.

If you encounter issues with ray installation, manually upgrade ray and pyarrow, and install pandas.

git clone https://github.com/meta-math/MetaMath.git
cd MetaMath
pip install -r requirements.txt

If you encounter a Ray installation problem, run:

pip install --upgrade ray
pip install --upgrade pyarrow
pip install pandas

The full MetaMathQA dataset is available on Hugging Face. You can access the training data files directly from the MetaMathQA repository to bootstrap mathematical questions for LLM training.

https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main

The MetaMathQA dataset is available on Hugging Face. You can access the full dataset files and directory structure via the official Hugging Face repository.

https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main

When running the data generation scripts, you can use the following arguments to control the behavior of the generation process:

• --num_repeat: Specifies the number of outputs to generate from ChatGPT for each input using temperature sampling.
• --part: Use this argument to specify a part number to avoid overwriting previously generated data.
• --cont: Use this argument to continue a previous generation procedure. If this is not used, the script will re-fetch data.