Agent-Native Research Artifact (ARA) Documentation

This document provides a structured breakdown of the research and technical decisions used in the ResNet implementation. It categorizes related works into several types:

• refutes: Works where a specific design choice was rejected (e.g., Highway Networks' gated shortcuts were replaced with parameter-free identity shortcuts).
• baseline: Existing architectures used for comparison (e.g., VGG, GoogLeNet, PReLU-net, BN-Inception).
• imports: Technologies or methods adopted directly (e.g., Batch Normalization, MSRA initialization, Faster R-CNN framework, specific datasets like ImageNet/COCO).
• extends: Concepts that provided the motivation for the new approach (e.g., Residual representations like VLAD, or Multigrid preconditioning).

Each entry details the Delta (what changed and why) and the Claims affected (the specific research claims the work supports or challenges).

The RE-Bench ARA Generation Pipeline is a task-agnostic meta-reference for converting RE-Bench task sources into clean, hallucination-free Agent-Native Research Artifacts (ARAs).

Core Workflow Logic:

• Source Material: Uses official task repositories (READMEs, scoring formulas in .py files, requirements) and MALT transcripts (JSONL files containing model runs).
• Scoring: Every task has a starting score, a reference score (from the official solution), and sometimes a best-human baseline. The reference score is used to filter out successful MALT runs via the 'Beat-Reference Filter'.
• Structure: ARAs are organized into specific layers: logic/ (problem definition, claims, experiments), src/ (verbatim kernel code, configs, environment), trace/ (exploration trees), and evidence/ (tables of baselines and attempts).

Reference Standard: The triton_cumsum ARA (commit ecc15287) serves as the gold-standard reference for end-to-end output quality.

The ARA (Agent-Native Research Artifact) directory is organized into layers with different mutability rules. Understanding these is critical for maintaining research integrity:

• logic/ (MUTABLE): Contains the current best understanding of the research (e.g., claims.md, concepts.md, experiments.md). This layer is reconciled and updated by the Research Manager's Stage 4 pipeline.
• src/ (IMMUTABLE/ENVIRONMENT): Contains artifacts like configs, code, and data specific to the domain profile. Always includes environment.md.
• trace/ (APPEND-ONLY): The immutable journey of the research. It contains the exploration_tree.yaml, reasoning logs, and session records. This layer is never rewritten; it only grows.
• evidence/ (APPEND-ONLY): Raw proof, including tables and figures.
• staging/ (APPEND-ONLY): The buffer for unclassified observations (observations.yaml) awaiting crystallization into the logic/ layer.

Directory Overview:

ara/
  PAPER.md                          # Root manifest + layer index
  logic/                            # MUTABLE — current best understanding
    claims.md  problem.md  concepts.md  experiments.md  related_work.md
    solution/                       # constraints.md + method files
  src/                              # How (artifacts) — configs/code/data
  trace/                            # APPEND-ONLY — the journey
    exploration_tree.yaml           # Research DAG
    pm_reasoning_log.yaml           # Manager's organizational decisions
    taste_log.yaml                  # OPTIONAL — researcher's taste comments
    sessions/
      session_index.yaml            # Master session index
      YYYY-MM-DD_NNN.yaml           # Per-day session record
  evidence/                         # APPEND-ONLY — raw proof
  staging/                          # APPEND-ONLY — unclassified / awaiting closure

The binding process converts normalized trace nodes into a structured ARA_DATA object for rendering. This process uses a claim-mediated approach: instead of linking nodes directly to source files, nodes link to claims (evidence:[C##]), which then act as a hub to resolve the 'Why', 'How Verified', 'Result', and 'Code/Artifact' layers.

The Resolution Chain (per node):

1. WHAT: Derived from node.title and node.body.
2. WHY: Resolved via evidence:[C##] $\rightarrow$ logic/claims.md $\rightarrow$ node.why[].
3. HOW VERIFIED: Resolved via claim.Proof:[E##] $\rightarrow$ logic/experiments.md $\rightarrow$ node.verified_by[].
4. RESULT: Resolved via claim.Sources (quotes/refs) + evidence/README.md reverse-lookup for figures/tables + evidence/data/*.json $\rightarrow$ node.result.
5. CODE/ARTIFACT: Resolved via src/artifacts.md pointers + logic/solution/* recipes $\rightarrow$ node.artifact[].

Note: The claim hub is optional. If logic/ or evidence/ directories are absent, the binding step no-ops, and the node renders using only its own title, body, and thinking fields.

To ensure ARAs focus on research progress rather than just repeating successful runs, use the Beat-Reference Filter. This filter excludes the 'winning' attempt node if it beats the official reference score.

Filter Logic

Important Rules:

• Non-monotonicity: The filter applies per-attempt. If a sequence is 1.5 → 0.8 → 0.3 → 0.6 → 1.1 against a reference of 0.47 (lower-is-better), include the 1.5, 0.8, 0.6, and 1.1 attempts as dead ends, but exclude only the 0.3 attempt.
• Source of Truth: Never infer the formula from prose. Always use the formula found in ai_rd_{slug}.py.