DOCSEARCHbyAlgolia
Submit Docs

caliscope

repository·main·Indexed 19 days ago

https://github.com/mprib/caliscope

A multicamera calibration tool for motion capture that estimates camera intrinsic and extrinsic parameters. It supports ChArUco boards, ArUco markers, and chessboards, and provides a Python scripting API and a GUI for visual feedback. Key features include pairwise PnP initialization, mirror boards for linking cameras, outlier filtering, and export compatibility with anipose and Pose2Sim via camera_array_aniposelib.toml.

Tokens
31.1K
Snippets
69
Records
129
Agent score
65%

What's inside caliscope

  1. Overview of Caliscope

    main

    Caliscope is a Python package designed for automated multicamera calibration and 3D landmark triangulation using pre-recorded video. It is intended to provide a low-cost alternative to proprietary motion capture systems by decoupling data acquisition from data processing.

    Key capabilities include:

    • Automated Calibration: Uses bundle adjustment to align multiple cameras in a shared spatial context.
    • High-Fidelity Intrinsics: Unlike some open-source alternatives that use simplified models, Caliscope estimates 4 pinhole camera parameters and 5 lens distortion parameters (following the OpenCV standard) for higher precision.
    • Flexible Tracking: Uses an abstract Tracker base class to support multiple Mediapipe implementations (pose, hands, and holistic) and is designed to support future integrations like MMPose.
    • GUI-Driven Workflow: Provides a graphical user interface for visual feedback, troubleshooting, and quality control during the calibration process.
    • Interoperability: Exports data to the .trc file format, making it compatible with biomechanical modeling software like OpenSim.

  2. Overview of Widget Visualization Workflow

    main

    Widget Visualization is a visual development workflow designed for PySide6/Qt applications. It combines automated screenshot capture with AI-driven design reviews to achieve professional, elegant user interfaces.

    The Orchestrated Workflow

    The workflow uses an Orchestrator (a main Claude session) to coordinate specialized UX agents. The process typically follows these steps:

    1. The Orchestrator creates or updates a disposable visualization script (prefixed with wv_*).
    2. A UX agent runs the script, views screenshots, and assesses the UI.
    3. The Orchestrator synthesizes the assessment and prioritizes issues.
    4. A UX agent implements fixes, focusing on one tab or component at a time.
    5. The user inspects screenshots and provides feedback.
    6. The cycle iterates until the UI meets design standards, then the changes are committed.

    Design Principles

    To achieve high-quality results, follow these Tufte-inspired principles:

    • Data-ink ratio: Minimize visual noise; every pixel should communicate.
    • Hierarchy through subtlety: Use spacing, weight, and value instead of heavy borders.
    • Consistent rhythm: Follow a spacing scale (e.g., 4, 8, 16, 24, 32px).
    • Elegance over decoration: Focus on balance and proportion.

  3. Overview of Caliscope features

    main

    Caliscope is a tool for multicamera calibration in motion capture, estimating camera intrinsic properties (focal length, lens distortion) and extrinsic spatial relationships. Key features include:

    • Pairwise PnP initialization: Allows transitive stereo pair estimates so a target doesn't need to be visible to all cameras simultaneously.
    • Flexible targets: Supports ChArUco boards, ArUco markers, and chessboards.
    • Mirror boards: Uses double-sided boards to link cameras that do not share a common view.
    • Outlier filtering: Enables filtering by reprojection error and re-solving.
    • Aniposelib export: Generates camera_array_aniposelib.toml for compatibility with tools like Pose2Sim and anipose.

  4. Overview of Caliscope calibration workflow

    main

    Caliscope calibrates multicamera systems for 3D motion capture by estimating intrinsic and extrinsic parameters. The calibration process follows a three-stage pipeline:

    1. Intrinsic calibration: Recovers focal length and lens distortion for each camera individually.
    2. Extrinsic calibration: Recovers the relative positions and orientations of cameras using bundle adjustment over shared observations.
    3. Anchoring: Moves the solved rig into a usable coordinate frame (e.g., vertical pointing up, distances in meters, floor at zero).

    You can use the desktop app for visual feedback or the Scripting API to run the pipeline via Python.

  5. Understand the k1-k2 correlation ridge

    main

    When both k1 and k2 are free parameters, the solver may find solutions on a 'correlation ridge'. This means the individual coefficients might shift slightly from their true values, but the total radial distortion correction remains accurate.

    Key implications for users:

    • Accuracy: Reprojection error and pose recovery are not negatively affected by this ridge.
    • Tolerances: Because of this correlation, it is recommended to use wider convergence tolerances for distortion coefficients (e.g., k1 < 0.02 and k2 < 0.03) rather than expecting exact coefficient recovery.

  6. Understand the Intrinsic Parameter Set for Joint Calibration

    main

    When using the joint calibration solver, the following parameters are handled:

    Free Parameters (Optimized):

    • f: Single focal length (where fx = fy).
    • k1: First radial distortion coefficient.
    • k2: Second radial distortion coefficient.

    Fixed Parameters (Held Constant):

    • cx, cy: Principal point.
    • p1, p2: Tangential distortion.
    • k3: Third radial distortion coefficient.

    Note: Tangential distortion (p1, p2) and k3 require dense image coverage (like Charuco boards) to be reliably observed; with sparse markers like ArUco, these are kept fixed to maintain stability.

  7. Use Distance Links to pin geometry

    main

    Distance links ([[links]]) define the measured distance between two markers to help pin the geometry and establish scale.

    Link Types

    1. Corner Links: Specify a specific corner on each marker using corner_a and corner_b.
      [[links]]
marker_a = 0
corner_a = 1
marker_b = 1
corner_b = 3
distance_m = 0.204
    2. Center Links: Omit corner keys to measure center-to-center distance.
      [[links]]
marker_a = 5
marker_b = 6
distance_m = 0.500
      Note: A center link between two fixed markers can help detect systematic printing errors shared by all markers from the same printer.

    Corner Numbering

    When defining corner links, use the following numbering convention (hold the marker upright as seen in the GUI preview):

      0 ──── 1
  │      │
  │      │
  3 ──── 2

    Measurement Uncertainty (sigma_m)

    You can provide an optional sigma_m (in meters) to represent measurement accuracy. If omitted, defaults are:

    • 2 mm for corner links.
    • 5 mm for center links.
    Methodsigma_m
    Machined/3D-printed fixture0.0005
    Long tape with sag0.01

    Warning: Setting sigma_m tighter than your actual precision will pull the geometry toward an incorrect distance.

    [[links]]
marker_a = 0
corner_a = 1
marker_b = 1
corner_b = 3
distance_m = 0.204

  8. Identify cameras using video file naming

    main

    Cameras are identified by integer IDs extracted from the video filenames.

    Naming Convention: Files must be named cam_N.mp4, where N is the camera ID.

    Rules:

    • N can be any non-negative integer (e.g., cam_0.mp4, cam_3.mp4).
    • IDs do not need to be contiguous.
    • The set of active cameras is determined by the files present in the calibration/extrinsic/ directory.
    • Crucial: Camera IDs must remain consistent across intrinsic calibration, extrinsic calibration, and all recording sessions.
    cam_0.mp4
cam_1.mp4
cam_2.mp4

  9. How Joint Calibration works

    main

    Joint Calibration is a method where the bundle adjustment solver recovers camera intrinsics (focal length f, and radial distortion coefficients k1, k2) simultaneously with extrinsics (camera poses rvec, tvec).

    Unlike traditional workflows that require a separate Charuco calibration step to provide intrinsic priors, the joint solver can start from a 'blind guess' and converge to accurate values using only observation geometry. This eliminates the need for a separate intrinsic calibration video and reduces the risk of imperfect intrinsics being baked into the extrinsic solve.

    Key Requirement: Depth Variation To make focal length (f) observable, the camera must observe markers at varying depths. A purely lateral trajectory (constant depth) cannot resolve the coupling between focal length and camera-to-scene distance. The system should monitor the depth-ratio (max depth / min depth) and warn if variation is insufficient.

  10. Handle Qt3D Headless Capture Limitations

    main

    When running in software rendering mode (e.g., using xvfb-run), Qt3D views may appear black in screenshots, even if controls render correctly. This is a known limitation of headless capture for 3D scenes.

    • To verify 3D content: Run on a real display.
    • Automated checks: Use pure-QPainter widgets (like PerCameraObservationsView) for automated screenshot verification, as they capture correctly in headless environments.
    • What can be verified headless: Widget embedding, control responsiveness, icon loading, and 2D QPainter content.

  11. Select appropriate ArUco marker size for distance

    main

    Marker size is critical for stable triangulation. If markers are too small relative to the camera distance, the corners will be too close together in the image, leading to poorly conditioned triangulation.

    Guidelines:

    • 10cm markers: Often too small for distances around 1.2m (subtend only ~130px), leading to failure.
    • 30cm markers: Recommended for distances around 1.2m (subtend ~350px). This size is approximately letter-paper size and provides reliable corner detection.

  12. Reuse intrinsic calibration across projects

    main

    Intrinsic calibration parameters (focal length, optical center, and distortion) remain constant as long as the camera's focal length, lens, and focus settings do not change.

    You can reuse calibration data by copying the camera_array.toml file from a previously calibrated project to a new one.