parakeet-mlx

For real-time transcription, use the transcribe_stream context manager. This allows you to feed audio chunks into the model and access both finalized and draft tokens.

from parakeet_mlx import from_pretrained
from parakeet_mlx.audio import load_audio
import numpy as np

model = from_pretrained("mlx-community/parakeet-tdt-0.6b-v3")

# context_size: (left_context, right_context) frames
with model.transcribe_stream(context_size=(256, 256)) as transcriber:
    audio_data = load_audio("audio_file.wav", model.preprocessor_config.sample_rate)
    chunk_size = model.preprocessor_config.sample_rate  # 1 second chunks

    for i in range(0, len(audio_data), chunk_size):
        chunk = audio_data[i:i+chunk_size]
        transcriber.add_audio(chunk)

        result = transcriber.result
        print(f"Current text: {result.text}")
        # Use transcriber.finalized_tokens or transcriber.draft_tokens for granular access

Streaming Parameters

• context_size: Tuple of (left_context, right_context) frames. Controls the attention window.
• depth: Number of encoder layers that preserve exact computation across chunks. Higher depth increases computational consistency with non-streaming mode. Default is 1.
• keep_original_attention: Boolean. If False (recommended), switches to local attention for streaming. If True, keeps original attention.

with model.transcribe_stream(context_size=(256, 256)) as transcriber:
    transcriber.add_audio(chunk)
    print(transcriber.result.text)

The Python API provides several ways to interact with the models, from high-level transcription to low-level log-mel processing.

Basic Transcription

Use from_pretrained to load a model and .transcribe() to get results.

from parakeet_mlx import from_pretrained

model = from_pretrained("mlx-community/parakeet-tdt-0.6b-v3")
result = model.transcribe("audio_file.wav")

print(result.text)

Accessing Timestamps

The result.sentences attribute returns a list of AlignedSentence objects containing timing information.

from parakeet_mlx import from_pretrained

model = from_pretrained("mlx-community/parakeet-tdt-0.6b-v3")
result = model.transcribe("audio_file.wav")

# Returns [AlignedSentence(text="...", start=..., end=..., duration=..., tokens=[...])]
print(result.sentences)

Advanced Transcription Configurations

Chunking

For long audio files, specify chunk_duration and overlap_duration.

result = model.transcribe("audio_file.wav", chunk_duration=120.0, overlap_duration=15.0)

Beam Decoding

Requires DecodingConfig and Beam (though Beam is often implied by the config structure).

from parakeet_mlx import from_pretrained, DecodingConfig

model = from_pretrained("mlx-community/parakeet-tdt-0.6b-v3")
config = DecodingConfig(
    decoding={"beam_size": 5, "length_penalty": 0.013, "patience": 3.5, "duration_reward": 0.67}
)
result = model.transcribe("audio_file.wav", decoding_config=config)

Sentence Splitting

Control how sentences are segmented using SentenceConfig.

from parakeet_mlx import from_pretrained, DecodingConfig, SentenceConfig

model = from_pretrained("mlx-community/parakeet-tdt-0.6b-v3")
config = DecodingConfig(
    sentence=SentenceConfig(max_words=30, silence_gap=5.0, max_duration=40.0)
)
result = model.transcribe("audio_file.wav", decoding_config=config)

Local Attention

To reduce memory usage for long audio, switch the attention model.

model.encoder.set_attention_model("rel_pos_local_attn", (256, 256))
result = model.transcribe("audio_file.wav")

The DecodingConfig class controls how the model decodes the acoustic features into text. It supports two primary modes:

1. Greedy Decoding: Uses the Greedy class. Fast, but less accurate as it picks the single most likely token at each step.
2. Beam Search: Uses the Beam class. More accurate, exploring multiple hypotheses simultaneously.

Beam Configuration Options:

• beam_size (int): Number of hypotheses to maintain (default: 5).
• length_penalty (float): Penalty applied to longer sequences (default: 1.0).
• patience (float): Controls how long the beam search continues (default: 1.0).
• duration_reward (float): TDT-only parameter to reward duration predictions (default: 0.7).

from parakeet_mlx import DecodingConfig, Beam, Greedy

# For Beam Search
config = DecodingConfig(decoding=Beam(beam_size=10, length_penalty=0.5))

# For Greedy Search
config = DecodingConfig(decoding=Greedy())

The following options are available for the parakeet-mlx CLI. Many can also be set via environment variables.

The parakeet_mlx.alignment module provides several dataclasses to represent structured transcription results, moving from individual tokens to full sentences and complete results.

• AlignedToken: Represents a single word or unit of speech.
• AlignedSentence: A collection of AlignedToken objects representing a sentence. It automatically calculates its own start, end, duration, and an aggregate confidence score (using the geometric mean of token confidences).
• AlignedResult: The top-level object containing the full transcribed text and a list of AlignedSentence objects. It provides a .tokens property to flatten all tokens from all sentences into a single list.

from parakeet_mlx.alignment import AlignedToken, AlignedSentence, AlignedResult

# Example of the hierarchy
token = AlignedToken(id=1, text="Hello", start=0.0, duration=0.5, confidence=0.9)
sentence = AlignedSentence(text="Hello world.", tokens=[token, ...])
result = AlignedResult(text="Hello world.", sentences=[sentence])

# Accessing all tokens in a result
all_tokens = result.tokens

The get_logmel function transforms a raw audio MLX array into a log-Mel spectrogram based on the provided PreprocessArgs.

Workflow:

1. Padding: If args.pad_to is set, the audio is padded to that length.
2. Pre-emphasis: If args.preemph is provided, it applies a high-pass filter.
3. STFT: Applies a Short-Time Fourier Transform using the specified window type (hann, hamming, blackman, or bartlett).
4. Mel Filterbank: Applies Mel filterbanks to the magnitude spectrum.
5. Log Scaling: Applies a log transform.
6. Normalization: Normalizes the output based on args.normalize (either "per_feature" or global normalization).

Returns an MLX array with shape (1, n_mels, time).

import mlx.core as mx
from parakeet_mlx.audio import load_audio, get_logmel, PreprocessArgs

# 1. Setup args
args = PreprocessArgs(
    sample_rate=16000,
    normalize="per_feature",
    window_size=0.02,
    window_stride=0.01,
    window="hann",
    features=80,
    n_fft=400,
    dither=0.0
)

# 2. Load audio
audio = load_audio("path/to/audio.wav", sampling_rate=16000)

# 3. Get log-mel spectrogram
log_mel = get_logmel(audio, args)