jiwer

By default, JiWER expects the reference and hypothesis files to have an equal number of lines. If your files have a different number of lines, use the -g or --global flag. This flag applies a global minimal alignment between the reference and hypothesis sentences before computing the WER.

# Use global alignment for files with unequal line counts
jiwer -r references.txt -h hypotheses.txt --global

The JiWER CLI is a Python tool used to compute the Word Error Rate (WER) or Character Error Rate (CER) of Automatic Speech Recognition (ASR) systems.

Input Format Requirements:

• Reference and hypothesis sentences must be stored in text files where each sentence is delimited by a new-line character.
• The reference and hypothesis files must contain an equal number of lines.
• Note: The CLI does not support custom pre-processing. You must manually pre-process your text files before running the command.

Basic Usage: To compute WER, provide paths to your reference and hypothesis files using the -r and -h flags.

To compute CER instead of WER, add the -c or --cer flag.

# Compute Word Error Rate (WER)
jiwer -r references.txt -h hypotheses.txt

# Compute Character Error Rate (CER)
jiwer -r references.txt -h hypotheses.txt --cer

When calculating Character Error Rate (CER), if your reference and hypothesis inputs contain a different number of sentences, use the contiguous variant to ensure the text is treated as a single unit.

• Use cer_default for standard character-level processing.
• Use cer_contiguous when the number of reference and hypothesis sentences differ.

When calculating Word Error Rate (WER), if your reference and hypothesis inputs contain a different number of sentences, the default transformations may fail or produce incorrect results. In these cases, you should use the contiguous variants of the transformations, which include a ReduceToSingleSentence step to flatten the input.

• Use wer_contiguous for standard word-level processing with mismatched sentence counts.
• Use wer_standardize_contiguous if you require text standardization (lowercase, contraction expansion, etc.) alongside sentence flattening.

Use visualize_alignment to create a human-readable string showing the alignment between reference and hypothesis pairs. It highlights substitutions (S), deletions (D), and insertions (I).

Arguments:

• output: The processed output from jiwer.process_words or jiwer.process_characters.
• show_measures (bool, default True): If True, includes summary statistics like WER, MER, WIL, and WIP in the output.
• skip_correct (bool, default True): If True, excludes sentences where the reference and hypothesis match perfectly.
• line_width (int, optional): If set, attempts to wrap long sentences into multiple lines to prevent horizontal scrolling.

import jiwer

out = jiwer.process_words(
    ["short one here", "quite a bit of longer sentence"],
    ["shoe order one", "quite bit of an even longest sentence here"],
)

print(jiwer.visualize_alignment(out))

Use visualize_error_counts to generate a summary report of the most frequent errors in your dataset. This is useful for identifying systematic errors in a speech-to-text model.

Arguments:

• output: The processed output from jiwer.process_words or jiwer.process_characters.
• show_substitutions (bool, default True): Whether to include the substitution report.
• show_insertions (bool, default True): Whether to include the insertion report.
• show_deletions (bool, default True): Whether to include the deletion report.
• top_k (int, optional): If provided, only shows the k most frequent errors for each category.

import jiwer

out = jiwer.process_words(
    ["short one here", "quite a bit of longer sentence"],
    ["shoe order one", "quite bit of an even longest sentence here"],
)
print(jiwer.visualize_error_counts(out))

Use collect_error_counts to extract raw frequency data of errors from a processed output object. It returns a three-tuple of dictionaries representing the counts for each error type.

Returns:

• substitutions: A dictionary where keys are (from_token, to_token) tuples and values are the frequency of that specific substitution.
• insertions: A dictionary where keys are the inserted tokens and values are their frequencies.
• deletions: A dictionary where keys are the deleted tokens and values are their frequencies.

Note: For WordOutput, tokens are joined by spaces; for CharacterOutput, they are joined by empty strings.

from jiwer import process_words, collect_error_counts

out = process_words(["hello world"], ["hello there world"])
substitutions, insertions, deletions = collect_error_counts(out)

JiWER provides convenience functions for Match Error Rate (mer), Word Information Preserved (wip), and Word Information Lost (wil). These all operate on a word-level basis using process_words internally.

Signatures:

• mer(reference, hypothesis, reference_transform=wer_default, hypothesis_transform=wer_default) -> float
• wip(reference, hypothesis, reference_transform=wer_default, hypothesis_transform=wer_default) -> float
• wil(reference, hypothesis, reference_transform=wer_default, hypothesis_transform=wer_default) -> float

Arguments:

• reference: The reference sentence(s) (str or List[str]).
• hypothesis: The hypothesis sentence(s) (str or List[str]).
• reference_transform: Transformation applied to reference. Defaults to wer_default.
• hypothesis_transform: Transformation applied to hypothesis. Defaults to wer_default.