CrisperWhisper

Repetition-loop detection and repair is enabled by default (hallucination_mitigation=True) on both backends.

Mechanism: After a greedy pass, the output is scanned for consecutive n-gram repetitions. If a loop is found, the output is rewound, one "escape" token is forced (the loop-starting token is banned for that step), and decoding resumes.

Customizing Thresholds: Default thresholds for triggering repair:

• 1-gram (unigrams): 8 repeats
• 2-gram (bigrams): 8 repeats
• 3-gram (trigrams): 4 repeats
• 4-gram: 3 repeats
• 5-gram: 3 repeats

You can disable this per call with hallucination_mitigation=False or use the lower-level generate_with_repair API to provide custom detect_reps thresholds.

# Disable mitigation per call
result = model.transcribe("audio.wav", hallucination_mitigation=False)

# Advanced: Custom thresholds via low-level API
from crisperwhisper.hallucination import generate_with_repair

gen_ids, n_repairs = generate_with_repair(
    engine, features, prompt_tokens,
    detect_reps={1: 10, 2: 6, 3: 3, 4: 2, 5: 2},  # custom thresholds
    keep_reps=1,
    max_repairs=3,
)

CrisperWhisper 2.0 is designed around two interchangeable backends that run the same high-level algorithms (longform, hallucination repair, and word timing) but differ in execution speed and capabilities:

1. CTranslate2 (ct2): A high-performance backend optimized for speed. It supports advanced features like speculative decoding and dual-mode transcription (processing verbatim and intended text in a single batched pass). It is the recommended backend for production use on NVIDIA GPUs.
2. Transformers: A portable backend using pure PyTorch. It is slower but highly compatible across different hardware (macOS, Windows, CPU). Note that for word-level timestamps, it uses attn_implementation="eager" to access cross-attention weights.

When using backend="auto", the library will attempt to use ct2 if the [ct2] extra is installed, otherwise it defaults to transformers.

To use CrisperWhisper, import CrisperWhisperModel. You can use the default large model or specify a size like turbo, medium, or small.

Verbatim Transcription (Default)

Transcribes exactly what was said, including fillers, repetitions, stutters, and vocal events.

Intended Transcription

Produces a clean, readable version of the speech, formatting numbers, dates, and emails.

Word-level Timestamps

Enables precise start and end times for every word.

Verbatimize

Upgrades an existing clean transcript by inserting the actual disfluencies present in the audio.

from crisperwhisper import CrisperWhisperModel

# Initialize the model (defaults to nyralabs/CrisperWhisper2.0_large)
model = CrisperWhisperModel() 
# or pick a size: CrisperWhisperModel("turbo")

# 1. Verbatim transcription (default)
result = model.transcribe("meeting.wav", language="en")
print(result.text)

# 2. Intended: the clean, readable version
clean = model.transcribe("meeting.wav", language="en", mode="intended")

# 3. Word-level timestamps
result = model.transcribe("meeting.wav", language="en", word_timestamps=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}")

# 4. Verbatimize: upgrade an existing clean transcript
result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large
# or pick a size: CrisperWhisperModel("turbo")  # turbo / medium / small

# Verbatim transcription (default): every filler, repetition, stutter,
# false start, and vocal event
result = model.transcribe("meeting.wav", language="en")
print(result.text)

# Intended: the clean, readable version
clean = model.transcribe("meeting.wav", language="en", mode="intended")

# Word-level timestamps
result = model.transcribe("meeting.wav", language="en", word_timestamps=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}")

# Verbatimize: upgrade an existing clean transcript with the
# disfluencies that are actually in the audio
result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

The core crisperwhisper package does not include an inference backend by default. You must install one or both using extras via pip:

• CTranslate2 ([ct2]): Fastest backend, supports speculative decoding. Requires an NVIDIA driver (CUDA libraries are provided via pip). Works on Linux x86_64.
• Transformers ([transformers]): Portable PyTorch backend. Works anywhere torch runs (macOS, Windows, CPU). Requires torch>=2.4.
• Both ([all]): Installs both backends.

Important Notes:

• Do not install faster-whisper or upstream ctranslate2 alongside [ct2], as they will overwrite the specialized fork.
• For Intel Macs where torch>=2.4 is unavailable, you can install a compatible pair manually: pip install crisperwhisper "transformers==4.49.*" "torch==2.2.*"

pip install crisperwhisper[ct2]            # CTranslate2 (fast, speculative decoding)
pip install crisperwhisper[transformers]   # pure torch + HuggingFace Transformers
pip install crisperwhisper[all]            # both backends

If you are building a new inference backend for CrisperWhisper, you must implement the EngineProtocol. This is a typing.Protocol used for structural typing, meaning engines are checked against this interface without requiring explicit inheritance or runtime registration.

All shared CrisperWhisper algorithms (such as prompt building, longform strategies, word timing, and hallucination repair) are written against this protocol. The protocol covers four main functional areas: tokenizer/special-id state, feature extraction, decoding, and cross-attention (for word timing).

Optional Capabilities

Some engines implement additional capabilities that are detected via hasattr rather than being part of the formal protocol:

• generate_with_attention: Used by SpeculativeDecoder to route word-timing through speculative capture.
• generate_dual_greedy: Used by CT2Engine to power transcribe_dual functionality.