Qwen3-ASR Documentation

Qwen3-ASR is a family of speech recognition models. It includes:

• Qwen3-ASR-1.7B & Qwen3-ASR-0.6B: All-in-one models supporting language identification and ASR for 52 languages and dialects (including 30 languages and 22 Chinese dialects). They support both offline and streaming inference for speech, singing voices, and songs with BGM.
• Qwen3-ForcedAligner-0.6B: A non-autoregressive (NAR) model that supports timestamp prediction for arbitrary units in up to 5 minutes of speech across 11 languages.

For multi-GPU training, use torchrun to launch the qwen3_asr_sft.py script. Set CUDA_VISIBLE_DEVICES to specify which GPUs to use and --nproc_per_node to the number of GPUs.

export CUDA_VISIBLE_DEVICES=0,1
torchrun --nproc_per_node=2 qwen3_asr_sft.py \
  --model_path Qwen/Qwen3-ASR-1.7B \
  --train_file ./train.jsonl \
  --output_dir ./qwen3-asr-finetuning-out \
  --batch_size 32 \
  --grad_acc 4 \
  --lr 2e-5 \
  --epochs 1 \
  --save_steps 200

If you want to develop or modify the code locally, clone the repository and install in editable mode.

git clone https://github.com/QwenLM/Qwen3-ASR.git
cd Qwen3-ASR
pip install -e .
# For vLLM support:
# pip install -e ".[vllm]"

To avoid browser microphone permission issues when accessing the demo remotely, run the service over HTTPS using --ssl-certfile and --ssl-keyfile.

1. Generate a self-signed certificate:

openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=localhost"

2. Run the demo with SSL flags:

qwen-asr-demo --asr-checkpoint Qwen/Qwen3-ASR-1.7B --backend transformers --cuda-visible-devices 0 --ip 0.0.0.0 --port 8000 --ssl-certfile cert.pem --ssl-keyfile key.pem --no-ssl-verify

openssl req -x509 -newkey rsa:2048 \
  -keyout key.pem -out cert.pem \
  -days 365 -nodes \
  -subj "/CN=localhost"

# Run with HTTPS
qwen-asr-demo \
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
  --backend transformers \
  --cuda-visible-devices 0 \
  --ip 0.0.0.0 --port 8000 \
  --ssl-certfile cert.pem \
  --ssl-keyfile key.pem \
  --no-ssl-verify

Install the required Python packages qwen-asr and datasets. It is highly recommended to install flash-attn (FlashAttention 2) to reduce GPU memory usage and accelerate training. FlashAttention 2 requires the model to be loaded in torch.float16 or torch.bfloat16 and compatible hardware.

If your machine has less than 96GB of RAM and many CPU cores, use MAX_JOBS=4 during the FlashAttention installation to prevent resource exhaustion.

To enable the vLLM backend for faster inference and streaming support, install the package with the [vllm] extra.

pip install -U qwen-asr[vllm]

You can resume training in two ways:

1. Explicitly: Provide the specific checkpoint path using --resume_from.
2. Automatically: Use --resume 1 to automatically find and load the latest checkpoint within the specified --output_dir.

# Option A: Explicit path
python qwen3_asr_sft.py \
  --train_file ./train.jsonl \
  --output_dir ./qwen3-asr-finetuning-out \
  --resume_from ./qwen3-asr-finetuning-out/checkpoint-200

# Option B: Automatic latest checkpoint
python qwen3_asr_sft.py \
  --train_file ./train.jsonl \
  --output_dir ./qwen3-asr-finetuning-out \
  --resume 1

vLLM provides efficient inference for Qwen3-ASR. It is recommended to use uv to install the nightly vLLM version with audio support.

Installation

uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
    --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
uv pip install "vllm[audio]"

Online Serving

Deploy the model server using:

vllm serve Qwen/Qwen3-ASR-1.7B

uv venv
source .venv/bin/activate
uv pip install -U vllm --pre \
    --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
uv pip install "vllm[audio]"

# Serve the model
vllm serve Qwen/Qwen3-ASR-1.7B

A pre-built Docker image qwenllm/qwen3-asr is available. You must have the NVIDIA Container Toolkit installed to access the GPU.

To run a container with a local workspace mounted:

1. Set your local path and desired host port.
2. Run the docker run command.
3. Note that services inside the container must bind to 0.0.0.0 for port forwarding to work.

Container Management:

• Start/Re-enter: docker start qwen3-asr && docker exec -it qwen3-asr bash
• Remove: docker rm -f qwen3-asr

LOCAL_WORKDIR=/path/to/your/workspace
HOST_PORT=8000
CONTAINER_PORT=80
docker run --gpus all --name qwen3-asr \
    -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
    --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-ASR \
    --shm-size=4gb \
    -it qwenllm/qwen3-asr:latest

To reduce GPU memory usage and accelerate inference (especially for long inputs), install FlashAttention 2. Note that FlashAttention 2 requires the model to be loaded in torch.float16 or torch.bfloat16.

pip install -U flash-attn --no-build-isolation

If your machine has less than 96GB of RAM and many CPU cores, use:

MAX_JOBS=4 pip install -U flash-attn --no-build-isolation

Run the qwen3_asr_sft.py script to start fine-tuning on a single GPU. Use the following arguments:

• --model_path: Path to the base model.
• --train_file: Path to your JSONL training file.
• --output_dir: Directory where checkpoints will be saved.
• --batch_size: Training batch size.
• --grad_acc: Gradient accumulation steps.
• --lr: Learning rate.
• --epochs: Number of training epochs.
• --save_steps: Interval for saving checkpoints.
• --save_total_limit: Maximum number of checkpoints to keep.

python qwen3_asr_sft.py \
  --model_path Qwen/Qwen3-ASR-1.7B \
  --train_file ./train.jsonl \
  --output_dir ./qwen3-asr-finetuning-out \
  --batch_size 32 \
  --grad_acc 4 \
  --lr 2e-5 \
  --epochs 1 \
  --save_steps 200 \
  --save_total_limit 5