TorchLens

The following scripts demonstrate specific intervention patterns and advanced features within TorchLens:

• Basic Workflow: 01_first_five_minutes.py covers minimal capture, discovery, forking, zero-ablation, and replay.
• Targeted Intervention:
  • 02_exact_site_after_discovery.py: Use find_sites to discover labels, then target a specific site.
  • 09_submodule_discover_first.py: Use tl.in_module for discovery scoped to a specific submodule.
• Patching & Steering:
  • 03_out_patching_paired_prompt.py: Perform clean-versus-corrupted out patching.
  • 14_per_position_steering.py: Steer using a per-position direction tensor.
• Hooks & Attachments:
  • 04_sticky_hooks_multiple_engines.py: Reuse sticky hooks with replay and rerun.
  • 05_set_vs_attach_hooks.py: Compare static set replacements against sticky hook recipes.
  • 08_live_post_hooks_during_capture.py: Execute hooks during the original capture process.
  • 11_sae_attachment.py: Attach an SAE-style nn.Module using splice_module.
  • 12_linear_probe_attachment.py: Collect linear-probe readouts via hooks.
  • 17_raw_forward_hook_replacement.py: Replace module outputs using raw PyTorch register_forward_hook.
• Data Management & Comparison:
  • 06_chunked_batching.py: Append compatible chunks for memory-constrained evaluation.
  • 07_bundle_comparison.py: Compare multiple Trace objects within a Bundle.
  • 13_paired_prompt_3plus.py: Compare three or more prompt variants in a Bundle.
  • 15_publishing_for_reproducibility.py: Save and load portable .tlspec/ files.
• Advanced Replay & Shortcuts:
  • 10_post_hoc_replay_generation_trace.py: Replay over captured generation-style traces.
  • 16_pearl_style_tl_do.py: Use the top-level tl.do shortcut.

The Menagerie crawler is a deterministic Python-driven system designed to execute models and award runs based on verified execution. It is governed by strict invariants to ensure data integrity and prevent unauthorized side effects.

Key Constraints:

• No TorchLens: The crawler must not import, trace, render, or validate traces using TorchLens. This is enforced by static and runtime tripwires.
• Deterministic Execution: The Python driver is the sole authority for awarding runs. All inherited claims (source, recipe, etc.) are treated as untrusted hints and must be re-verified.
• Isolated Environments: Models run in thick, sequential conda environments. The driver never invokes a shell; it passes an argv vector to a subprocess with offline flags and empty caches.
• Single-Writer Topology: The system is not distributed. It uses a single-writer driver and one execution worker at a time, managed via a worker.lock file.

The Visual Audit Pack provides a comprehensive suite of visual demonstrations for TorchLens features. The coverage matrix serves as an inventory of these demonstrations, organized by section (A-L) and specific visual axes.

Key functional areas covered in the audit include:

• Graph Anatomy & Structure: Baseline renders, nesting, buffers, multiple I/O, and edge multiplicity.
• Layout & Direction: Controlling graph direction (bottomup, topdown, leftright), sibling order, and node placement (dot vs rank).
• Container & Module Control: Managing container visibility (show_containers), module focus (module=), call depth (vis_call_depth), and custom collapsing logic (collapse_fn, skip_fn).
• Loop & Recurrence Handling: Visualizing unrolled vs. rolled loops, RNN/LSTM/GRU cell structures, and pass-count sweeps.
• Collapse & Simplification: Using collapse modes (none, auto, max), folding repeats (fold_repeats), and ellipsis grammar.
• Node Customization & Overlays: Using node_mode (e.g., vision, attention, profiling), applying metric/diagnostic overlays, and customizing labels (node_label_fields).
• Theming & Aesthetics: Applying themes like torchlens, paper, dark, colorblind, or high_contrast via vis_theme.
• Backward Pass Visualization: Using draw_backward for grad_fn graphs and draw_combined to show forward and backward passes together.
• Intervention & Conditionals: Visualizing if/else logic, intervention modes (node_mark vs as_node), and downstream impact cones (vis_show_cone).
• Diagnostics: Using preview_fastlog to predict predicate filtering and bundle_diff to see per-node deltas between clean and intervened traces.

Use the intervene= argument in tl.trace when you need an edited value to affect the actual downstream execution of the model during the trace. For example, you can use tl.zero_ablate() to zero out activations at a specific point.

If you are performing post-hoc experiments on an existing trace (where the forward pass has already completed), do not use intervene=. Instead, use trace.fork(), trace.set(...), trace.attach_hooks(...), and trace.replay().

import torch
from torch import nn
import torchlens as tl

model = nn.Sequential(nn.Linear(4, 4), nn.ReLU(), nn.Linear(4, 2)).eval()
x = torch.randn(2, 4)

# Intervention happens during the trace and affects downstream layers
patched = tl.trace(
    model,
    x,
    save=tl.func("relu"),
    intervene=tl.when(tl.func("relu"), tl.zero_ablate()),
)

assert torch.count_nonzero(patched.find_sites(tl.func("relu")).first().out) == 0

The menagerie/crawler-pipeline repository is organized into specific directories for committed content, public releases, and local/ignored state.

Committed Content (Public at Merge)

• adapters/, ports/, patches/: Code used to interface with or modify models.
• evidence/: JSON files containing grounding for models/families.
• source_manifests/: Manifests for model sources.
• records/: JSONL files for intake, models, attempts, gates, and operational data.
• views/: Generated views like current-models/*.jsonl and status-summary.json.

Durable Public Release Store

Legally redistributable source archives and package artifacts are stored in a content-addressed store keyed by SHA-256. The repository commits the manifest (digest, size, media type, upstream URL, etc.) but not the full bytes.

Private Restricted Mirror

GPL/AGPL and no-license upstream bytes are kept in a private mirror. The public repository only contains metadata (URL, revision, hash, byte count, license finding, and fetch recipe). Full bytes are never committed to the public repo.

Local and Gitignored (Not in Repo)

• .crawl-local/: Contains state.sqlite, logs/, source-cas/, artifact-cas/, envs/, caches/, scratch/, and locks/.

In TorchLens, Generators (such as those used for RNG or state management) are subject to a strict classifier that ensures they are 'closed under the return value'. This means any host-scalar return is witnessed across all receiver classes.

Key behaviors include:

• Entropy Ceilings: Methods like seed() return fresh OS entropy as a Python int on any receiver, creating an entropy ceiling.
• Replayability: initial_seed() is only a replayable, read-only operation on a proven process/device default. On any other receiver (like an instance or clone), it 'ceilings' (returns a non-replayable value) because clone lineage is not tracked. This is a 'fail-closed' design.
• State Management: clone_state() and graphsafe_get_state() are structural only. Mutations like manual_seed, set_state, set_offset, and graphsafe_set_state remain inert instance state on non-default receivers, whereas they act as host mutations on a default receiver.
• Routing Cache: Membership in a 'default-receiver' is a dynamically re-resolved routing cache. If a receiver miss occurs, the classifier re-resolves the currently populated torch.{cuda,xpu,mtia}.default_generators without requiring new imports or device initialization.

TorchLens ensures deterministic replay by monitoring and inventorying all sources of entropy and non-deterministic state within a model. The system uses a 'frozen contract' approach to track:

1. Random Number Generators (RNGs): Includes random.Random, numpy.Generator, torch.Generator, and various device-specific RNG namespaces (torch.cuda, torch.mps, etc.).
2. Entropy Sources: Direct monitoring of os.urandom, os.getrandom, secrets, and uuid.uuid4.
3. Clock/Time Sources: Tracking calls to time.*, datetime.datetime.now, and os.times to ensure time-dependent logic is captured.
4. State Inventory: A cycle-safe, non-executing walk of the model's object graph (using gc.get_referents) to find stashed RNGs, even if they are hidden in private __slots__, closures, or custom container subclasses.

If a source of non-determinism is encountered that cannot be fully witnessed or inventoried, the capture is marked as INCOMPLETE (e.g., inventory_opaque_container or inventory_budget_exhausted) rather than falsely claiming it is verified.