iOS Simulator Skill

repository·main·Indexed 22 days ago

https://github.com/conorluddy/ios-simulator-skill

An iOS development automation skill for Claude Code (v1.5.0) designed to build, test, and interact with iOS simulators. It provides 27 specialized scripts for AI agents, utilizing an accessibility-driven navigation strategy to interact with the UI tree for token efficiency. Key features include automated Xcode project builds and tests via build_and_test.py, real-time log monitoring with log_monitor.py, semantic UI interaction with navigator.py and screen_mapper.py, and simulator control for appearance, GPS location, and gestures.

Tokens
13.4K
Snippets
66
Records
71
Agent score
78%

What's inside ios-simulator-skill

  1. Record and analyze hangs with hang_watcher.py

    main

    The hang_watcher.py script (HangBuster) records and summarizes os_log hang events using two modes:

    1. Session Mode (Agent-Native)

    Designed for interactive debugging. It uses a detached recorder that normalizes and clusters events to provide a token-efficient summary.

    Workflow:

    1. Start: SID=$(python scripts/hang_watcher.py --start --min-hang-ms 200)
    2. Interact: Perform actions in the simulator.
    3. Stop: python scripts/hang_watcher.py --stop $SID (returns an L1 summary).
    4. Drill Down: python scripts/hang_watcher.py --get-details $SID --cluster 1 (returns L2 details).
    5. Compare: python scripts/hang_watcher.py --diff $SID_BASELINE $SID (regression report).

    2. Raw Capture Mode

    Skips clustering for full-fidelity log dumping, ideal for jq exploration.

    Workflow:

    1. Start: SID=$(python scripts/hang_watcher.py --start --raw-capture --max-size-mb 5)
    2. Stop: python scripts/hang_watcher.py --stop $SID
    3. Explore: Use zcat and jq on the resulting .gz file in ~/.ios-simulator-skill/sessions/$SID/raw.ndjson.gz.

    Key Options:

    • --start: Begins a new session.
    • --stop <SESSION_ID>: Ends session and summarizes.
    • --get-details <SESSION_ID>: Provides deeper inspection.
    • --raw-capture: Enables raw NDJSON capture.
    • --max-size-mb: Sets a cap on the raw capture size.
    • --min-hang-ms: Minimum duration for a hang to be recorded.
    # Summarized mode workflow
    SID=$(python scripts/hang_watcher.py --start --min-hang-ms 200)
    # ... interact with simulator ...
    python scripts/hang_watcher.py --stop $SID
    python scripts/hang_watcher.py --get-details $SID --cluster 1
  2. How the navigation strategy works

    main

    The skill uses an accessibility-driven navigation strategy. Instead of using pixel coordinates or heavy image analysis, it prioritizes the accessibility tree. This provides structured data (element types, labels, frames, and tap targets) which is significantly more token-efficient and reliable for AI agents than screenshots.

    Priority Order for Navigation:

    1. screen_mapper.py: Use this to get a structured element list (highly efficient, ~10 tokens).
    2. navigator.py: Use semantic flags like --find-text, --find-type, or --find-id for direct interaction.
    3. Screenshots: Use only for visual verification, bug reports, or visual diffs. Screenshots are expensive (1,600–6,300 tokens) compared to the accessibility tree (10–50 tokens).
  3. Common Script Patterns and Options

    main

    The scripts in this project follow several consistent patterns:

    • Auto-UDID Detection: Most scripts automatically detect the booted simulator if the --udid flag is not provided.
    • Device Name Resolution: You can use human-readable device names (e.g., "iPhone 16 Pro") instead of UDIDs; scripts resolve these automatically.
    • Batch Operations: Use --all to target all simulators or --type <type> (e.g., --type iPhone) to filter by device type.
    • Output Formats: Use --json for machine-readable output suitable for CI/CD. The default is concise human-readable text.
    • Screenshot Sizing: To manage token usage, you can specify screenshot presets:
      • full: High detail (~5K tokens).
      • half: Default (~1.6K tokens).
      • quarter: Low detail (~800 tokens) for quick visual checks.
    • Help: Every script supports the --help flag for detailed options and usage examples.
  4. Typical Workflow for iOS Simulator Automation

    main

    To automate iOS simulator tasks, follow this standard sequence of operations:

    1. Verify environment: Ensure prerequisites are met. bash scripts/sim_health_check.sh
    2. Launch app: Start your application using its bundle ID. python scripts/app_launcher.py --launch com.example.app
    3. Analyze screen: Map the current UI to identify elements. python scripts/screen_mapper.py
    4. Interact: Perform actions like tapping buttons or entering text. python scripts/navigator.py --find-text "Button" --tap
    5. Verify: Run an accessibility audit to ensure UI compliance. python scripts/accessibility_audit.py
    6. Debug: If issues occur, capture the full application state. python scripts/app_state_capture.py --app-bundle-id com.example.app
    # Example sequence
    bash scripts/sim_health_check.sh
    python scripts/app_launcher.py --launch com.example.app
    python scripts/screen_mapper.py
    python scripts/navigator.py --find-text "Button" --tap
    python scripts/accessibility_audit.py
  5. Establish a Performance Baseline

    main

    Measure performance impact by capturing a screenshot of the initial state, running the app/test, and then comparing the state after a delay using scripts/visual_diff.py.

    # Capture initial state
    xcrun simctl io booted screenshot perf-before.png
    # Run performance test
    xcrun simctl launch booted com.example.app
    sleep 5
    xcrun simctl io booted screenshot perf-after.png
    python scripts/visual_diff.py perf-before.png perf-after.png
  6. Install the iOS Simulator Skill for Claude Code

    main

    You can install the skill using the Claude Code plugin marketplace or by cloning the repository directly into your Claude configuration directory.

    Run these commands within Claude Code:

    /plugin marketplace add conorluddy/ios-simulator-skill
    /plugin install ios-simulator-skill@conorluddy

    Via Git Clone

    Personal installation (available to all Claude Code sessions):

    git clone https://github.com/conorluddy/ios-simulator-skill.git ~/.claude/skills/ios-simulator-skill

    Project installation (available only within a specific project):

    git clone https://github.com/conorluddy/ios-simulator-skill.git .claude/skills/ios-simulator-skill

    After installation, restart Claude Code to load the skill.

  7. Prerequisites for iOS Simulator Skill

    main

    Before using the skill, ensure your environment meets the following requirements:

    • macOS 12+
    • Xcode Command Line Tools: Install via xcode-select --install
    • Python 3
    • IDB (Optional): Required for interactive features. Install via: brew tap facebook/fb && brew install idb-companion
    • Pillow (Optional): Required for visual diffs. Install via: pip3 install pillow
  8. Perform UI automation with idb ui commands

    main

    Use the idb ui command suite to interact with the iOS Simulator via accessibility. These commands allow you to inspect the UI tree, tap specific coordinates, swipe, enter text, and identify elements at specific points.

    Available UI Commands:

    CommandUsageDescription
    ui describe-allidb ui describe-all --json --nestedReturns the complete accessibility tree. Useful for auditing and finding element paths.
    ui tapidb ui tap <x> <y>Taps the screen at the specified X and Y coordinates.
    ui swipeidb ui swipe <x1> <y1> <x2> <y2>Performs a swipe gesture from $(x1, y1)$ to $(x2, y2)$.
    ui textidb ui text "<text>"Inputs the specified text into the currently focused element.
    ui describe-pointidb ui describe-point <x> <y> --jsonReturns the JSON representation of the element located at the specified coordinates.
    # Get accessibility tree
    idb ui describe-all --json --nested > tree.json
    
    # Basic interaction sequence
    idb ui tap 200 400
    idb ui text "username@example.com"
    idb ui tap 200 500
  9. Run Visual Regression tests

    main

    Compare the current UI against a baseline image to detect visual changes. First, capture a baseline.png, then capture the current.png after changes, and use scripts/visual_diff.py to find differences.

    # Baseline
    xcrun simctl io booted screenshot baseline.png
    
    # After changes
    xcrun simctl io booted screenshot current.png
    python scripts/visual_diff.py baseline.png current.png