DOCSEARCHbyAlgolia
Submit Docs

PhoWhisper Automatic Speech Recognition

repository·main·Indexed 42 days ago

https://github.com/vinairesearch/phowhisper

State-of-the-art Automatic Speech Recognition (ASR) fine-tuned for the Vietnamese language across various accents. Available in five model sizes and compatible with the Hugging Face transformers automatic-speech-recognition pipeline.

Tokens
635
Snippets
2
Records
2
Agent score
4%

What's inside PhoWhisper

  1. Run PhoWhisper models using Hugging Face Transformers

    main

    You can use PhoWhisper for Vietnamese automatic speech recognition by loading the models via the Hugging Face transformers library. The models are compatible with the automatic-speech-recognition pipeline.

    Note: Ensure your input audio files have a sampling rate of 16kHz.

    from transformers import pipeline

# Initialize the transcriber with a PhoWhisper model
transcriber = pipeline("automatic-speech-recognition", model="vinai/PhoWhisper-small")

# Run transcription on an audio file (must be 16kHz)
output = transcriber(path_to_audio_with_sampling_rate_16kHz)['text']

  2. Available PhoWhisper model versions

    main

    PhoWhisper is available in five sizes, fine-tuned on an 844-hour Vietnamese dataset. You can download these models from Hugging Face. The large version provides the best Word Error Rate (WER) performance across benchmark datasets like CMV–Vi, VIVOS, and VLSP 2020.

    | Model | #paras | CMV–Vi | VIVOS | VLSP 2020 Task-1 | VLSP 2020 Task-2 |
|---|---|---|---|---|---|
| [`vinai/PhoWhisper-tiny`](https://huggingface.co/vinai/PhoWhisper-tiny) | 39M |19.05 |10.41 |20.74 |49.85|
| [`vinai/PhoWhisper-base`](https://huggingface.co/vinai/PhoWhisper-base) | 74M |16.19 |8.46 |19.70| 43.01|
| [`vinai/PhoWhisper-small`](https://huggingface.co/vinai/PhoWhisper-small) | 244M |11.08 |6.33 |15.93 |32.96|
| [`vinai/PhoWhisper-medium`](https://huggingface.co/vinai/PhoWhisper-medium) | 769M |8.27 |4.97 |14.12 |26.85|
| [`vinai/PhoWhisper-large`](https://huggingface.co/vinai/PhoWhisper-large) | **1.55B** |**8.14** |**4.67** |**13.75** |**26.68**|