Lightning Whisper MLX

The batch_size parameter controls decoding throughput.

• Default value: 12
• Heuristic: Higher batch sizes increase throughput but consume more memory.
  • For smaller models: Use a higher batch_size.
  • For larger models: Use a lower batch_size.
• Constraint: Ensure the batch size does not exceed your Apple Silicon unified memory capacity.

Install the package using pip to get an optimized Whisper implementation for Apple Silicon.

pip install lightning-whisper-mlx

Use the LightningWhisperMLX class to perform speech-to-text transcription. You can specify the model size and quantization level during initialization. The transcribe method returns a dictionary containing the transcription results.

from lightning_whisper_mlx import LightningWhisperMLX

# Initialize the model
whisper = LightningWhisperMLX(model="distil-medium.en", batch_size=12, quant=None)

# Transcribe an audio file
text = whisper.transcribe(audio_path="/audio.mp3")['text']

print(text)

When initializing LightningWhisperMLX, you can select from several model variants and quantization levels to balance speed, accuracy, and memory usage.

### Supported Models
- `tiny`
- `small`
- `distil-small.en`
- `base`
- `medium`
- `distil-medium.en`
- `large`
- `large-v2`
- `distil-large-v2`
- `large-v3`
- `distil-large-v3`

### Supported Quantization
- `None` (Default)
- `4bit`
- `8bit`