DOCSEARCHbyAlgolia
Submit Docs

Ghost Pepper Documentation

repository·main·Indexed 25 days ago

https://github.com/matthartman/ghost-pepper

A private, on-device voice transcription tool for macOS utilizing Apple Silicon-optimized models for speech-to-text and LLM-based text cleanup. Features include support for Whisper, Parakeet, Qwen3-ASR, and Nemotron models, global hotkey transcription, and strict local-first privacy controls.

Tokens
2.7K
Snippets
3
Records
13
Agent score
85%

What's inside Ghost Pepper

  1. Implement MLX Nemotron backend in GhostPepper

    main

    To add NVIDIA Nemotron 0.6B ASR models as MLX-backed, streaming-only transcription options in GhostPepper, follow these integration steps:

    1. Update Catalog and Manager: Modify project.yml, GhostPepper/Transcription/SpeechModelCatalog.swift, and GhostPepper/Transcription/ModelManager.swift to include the new Nemotron descriptors and MLX backend identity.
    2. Configure Recording Sessions: Update GhostPepper/Transcription/RecordingSessionCoordinator.swift to ensure MLX selection creates independent incremental sessions that forward audio chunks and flush exactly once, avoiding offline transcription APIs.
    3. Add Dependencies: Pin the MLXAudioSwift dependency and implement the required backend adapter in ModelManager.
    4. Routing: Ensure both Nemotron models are routed exclusively through the recording streaming-session path.
    5. Verification: Run GhostPepperTests/SpeechTranscriberTests.swift, GhostPepperTests/ModelManagerTests.swift, and GhostPepperTests/RuntimeModelInventoryTests.swift to confirm integration.

  2. Qualify Nemotron MLX checkpoints

    main

    When validating the Nemotron streaming models, use the following Hugging Face repositories and verification steps:

    Models to Validate:

    • animaslabs/nemotron-speech-streaming-en-0.6b-mlx-8bit (English-only legacy checkpoint)
    • mlx-community/nemotron-3.5-asr-streaming-0.6b-8bit (Current 3.5 checkpoint)

    Validation Procedure:

    1. Load the model into a NemotronASRStreamSession.
    2. Transcribe local utterance fixtures through the streaming path.
    3. Capture the following metrics for reproducibility:
      • Model revision and commit ID
      • Hardware specifications
      • Utterance duration
      • Transcript output
      • Wall time and Real-time factor (RTF)
      • Peak memory usage

  3. Use Ghost Pepper for transcription and pasting

    main
    Ghost Pepper provides a global hotkey for quick speech-to-text. Hold the Control key to speak, and release it to automatically transcribe your speech and paste the text into the currently active text field.

  4. Add legacy Nemotron compatibility to MLXAudioSwift

    main

    If you are working with the older English-only Nemotron checkpoint, you must implement compatibility in MLXAudioSwift to handle its legacy NeMo configuration and packed pointwise-convolution layout:

    1. Configuration: Update NemotronASRConfig.swift to decode both current flattened Nemotron 3.5 configurations and the older nested NeMo configurations (specifically handling nested decoder.prednet, joint.jointnet, and vocabulary under joint).
    2. Optional Prompts: Make the prompt kernel optional to prevent the English-only checkpoint from inventing multilingual prompt parameters.
    3. Weight Sanitization: For legacy packed 8-bit pointwise Conv1d weights, dequantize them and reshape them from [out, in] to [out, 1, in] during the sanitization process to ensure they gain the Conv1d singleton kernel dimension.

  5. Install Ghost Pepper on macOS

    main

    To install Ghost Pepper, download the DMG, move it to your Applications folder, and grant the required Microphone and Accessibility permissions.

    Handling Gatekeeper warnings on macOS Sequoia: If you see an "Apple could not verify" warning, go to System Settings > Privacy & Security, scroll down, and click Open Anyway next to the Ghost Pepper message, then click Confirm.

    1. Download GhostPepper.dmg
2. Open the DMG, drag Ghost Pepper to Applications
3. Grant Microphone and Accessibility permissions when prompted

  6. Configure Accessibility permissions for managed devices

    main

    On managed macOS devices, IT admins can pre-approve the required Accessibility permission using an MDM profile (e.g., Jamf, Kandji) with a Privacy Preferences Policy Control (PPPC) payload using these values:

    • Permission: com.apple.security.accessibility
    • Bundle ID: Use the signed app's bundle identifier
    • Team ID: Use the signing team's Apple Developer team ID

  8. Run the Pre-Deploy Privacy and Security Gate

    main

    Before performing any public build, appcast update, GitHub release, or notarized DMG upload, you must execute the privacy and security preflight checks. This ensures that no sensitive data or security vulnerabilities are introduced into the release candidate.

    Follow these steps:

    1. Execute the local static preflight script.
    2. Run or review the Codex audit suite covering leaks, network egress, logs, sandboxing, and dependencies.
    3. Verify that no real user data (e.g., meeting transcripts, participant names, API keys, or local config) is present in any release inputs.
    4. Confirm all network paths are expected and that user content is not being sent over the network unexpectedly.
    5. Review new UI elements for potential private content exposure.
    ./scripts/privacy-security-preflight.sh

  9. Verify Ghost Pepper privacy and local execution

    main

    To verify that Ghost Pepper is running 100% locally with no unauthorized network calls, you can run a privacy audit using an AI assistant (like Claude Code) with access to the repository.

    Use the following prompt to audit the codebase:

    Review the Ghost Pepper codebase and confirm that all default features run 100% locally with no network calls. Specifically verify:

    1. Speech-to-text — transcription runs entirely on-device (WhisperKit, FluidAudio, Qwen3-ASR). No audio is sent to any server.
    2. Text cleanup — the local Qwen LLM cleans up transcriptions on-device. No text is sent to any API.
    3. Audio recording — mic and system audio capture use AVAudioEngine and ScreenCaptureKit only. No audio is streamed or uploaded.
    4. Meeting transcription & summarization — chunked transcription, markdown storage, and LLM summary generation all happen locally. No meeting content leaves the device.
    5. OCR & screen capture — Vision framework runs on-device. Screenshots are processed locally and never transmitted.
    6. File storage — all meeting notes, transcripts, and summaries are saved as local markdown files. No cloud sync, no remote backup.
    7. No third-party tracking SDKs — confirm there are no tracking SDKs (Firebase, Mixpanel, Sentry, Amplitude, PostHog, Datadog, Segment, etc.) anywhere in the codebase. Local-only usage counters (UserDefaults) power the in-app Usage report panel — verify they don't make network calls.

    For each item, check the relevant source files and confirm no URLSession, URLRequest, or HTTP URL strings exist in the core feature code. List any cloud-connected features separately and confirm they are all opt-in (disabled by default, require user-provided API keys).

    Output your findings as a checklist with pass/fail for each item.

    > Review the Ghost Pepper codebase and confirm that all default features run 100% locally with no network calls. Specifically verify:
>
> 1. **Speech-to-text** — transcription runs entirely on-device (WhisperKit, FluidAudio, Qwen3-ASR). No audio is sent to any server.
> 2. **Text cleanup** — the local Qwen LLM cleans up transcriptions on-device. No text is sent to any API.
> 3. **Audio recording** — mic and system audio capture use AVAudioEngine and ScreenCaptureKit only. No audio is streamed or uploaded.
> 4. **Meeting transcription & summarization** — chunked transcription, markdown storage, and LLM summary generation all happen locally. No meeting content leaves the device.
> 5. **OCR & screen capture** — Vision framework runs on-device. Screenshots are processed locally and never transmitted.
> 6. **File storage** — all meeting notes, transcripts, and summaries are saved as local markdown files. No cloud sync, no remote backup.
> 7. **No third-party tracking SDKs** — confirm there are no tracking SDKs (Firebase, Mixpanel, Sentry, Amplitude, PostHog, Datadog, Segment, etc.) anywhere in the codebase. Local-only usage counters (UserDefaults) power the in-app Usage report panel — verify they don't make network calls.
>
> For each item, check the relevant source files and confirm no `URLSession`, `URLRequest`, or HTTP URL strings exist in the core feature code. List any cloud-connected features separately and confirm they are all opt-in (disabled by default, require user-provided API keys).
>
> Output your findings as a checklist with pass/fail for each item.

  10. Verify release inputs for sensitive user data

    main

    Ensure that the following types of data are NOT included in tracked, modified, or untracked release inputs:

    • Granola cache files or imported meeting markdown
    • Meeting participant names from private tests
    • Transcripts, summaries, or people indexes from real meetings
    • Audio/video files, screenshots, OCR text, or debug logs
    • API keys, tokens, signing credentials, or local config

  11. Confirm expected network paths

    main

    Validate that network activity adheres to these constraints:

    • Core transcription, cleanup, OCR, local summaries, and local storage must NOT send user content over the network.
    • Cloud integrations must be opt-in and require user configuration.
    • Network paths for Sparkle, GitHub, and Hugging Face must be restricted to release, update, and model-download activities only.

  12. Configure Speech and Cleanup models

    main

    Ghost Pepper uses various local models for speech-to-text and text cleanup. Models download automatically and are cached locally. You can customize these in the app settings.

    Speech Models

    • Whisper tiny.en: Fastest, English only (~75 MB).
    • Whisper small.en (Default): Best accuracy, English only (~466 MB).
    • Whisper small (multilingual): Multi-language support (~466 MB).
    • Parakeet v3: 25 languages via FluidAudio (~1.4 GB).
    • Qwen3-ASR 0.6B int8: Highest multilingual quality, requires macOS 15+ (~900 MB).
    • Nemotron Speech Streaming 0.6B 8-bit: Low-latency English streaming via MLX Audio (~633 MB).
    • Nemotron 3.5 ASR Streaming 0.6B 8-bit: Low-latency multilingual streaming via MLX Audio (~721 MB).

    Cleanup Models

    • Qwen 3.5 0.8B (Default): Very fast (~1-2s, ~535 MB).
    • Qwen 3.5 2B: Fast (~4-5s, ~1.3 GB).
    • Qwen 3.5 4B: Full quality (~5-7s, ~2.8 GB).