ProcessBench: Identifying Process Errors in Mathematical Reasoning

Each record in the ProcessBench dataset follows a specific JSON structure used for identifying process errors in mathematical reasoning. Key fields include:

• id: Unique identifier for the sample.
• generator: The model used to generate the reasoning steps.
• problem: The mathematical problem text.
• steps: A list of strings representing the individual reasoning steps.
• final_answer_correct: A boolean indicating if the final answer is correct.
• label: An integer label associated with the sample.

{
  "id": "gsm8k-0",
  "generator": "Qwen2-7B-Instruct",
  "problem": "Sue lives in a fun neighborhood...",
  "steps": [
    "To find out how many more pink plastic flamingos were out than...",
    ...
  ],
  "final_answer_correct": false,
  "label": 1
}

You can access and preview the ProcessBench dataset using the Hugging Face datasets library. The dataset is hosted under the name Qwen/ProcessBench. To inspect a specific split (e.g., gsm8k), load the dataset and access individual entries. Each entry contains the problem statement, a sequence of reasoning steps, the correctness of the final answer, and a label.

import json
from datasets import load_dataset

dataset = load_dataset('Qwen/ProcessBench', split='gsm8k')
print(json.dumps(dataset[0], indent=2))