api4sensevoice

The Single Sentence Recognition API is a FastAPI-based service for transcribing audio files. You can start the server by running server.py.

By default, it runs on port 7000. You can customize the port and provide SSL certificate/key files via command-line arguments.

python server.py --port 8888 --certfile path_to_your_certfile --keyfile path_to_your_key

To install the project, clone the repository and set up a Conda environment with Python 3.10 and FFmpeg, then install the Python dependencies.

1. Clone the repository:

   git clone https://github.com/0x5446/api4sensevoice.git
   cd api4sensevoice

2. Create and activate the Conda environment:

   conda create -n api4sensevoice python=3.10
   conda activate api4sensevoice

3. Install FFmpeg and requirements:

   conda install -c conda-forge ffmpeg
   pip install -r requirements.txt

The WebSocket server (server_wss.py) provides real-time streaming recognition. By default, it runs on port 27000.

Configuration via CLI:

python server_wss.py --port 8888 --certfile path_to_your_certfile --keyfile path_to_your_key

Enabling Speaker Verification: To use speaker verification, you must provide reference audio files for the speakers:

1. Prepare WAV files: 16000 sampling rate, single channel, 16-bit width.
2. Place them in the speaker/ directory.
3. In server_wss.py, update the reg_spks_files list with the paths to your files:

   reg_spks_files = [
       "speaker/speaker1_a_cn_16k.wav"
   ]

   Any matching speaker file will allow ASR inference to proceed.

The server uses format_str_v3 to post-process the raw output from the SenseVoiceSmall model. This process:

1. Replaces language tokens (like <|zh|>, <|en|>) with a generic <|lang|> placeholder.
2. Maps emotion tokens (e.g., <|HAPPY|>) and event tokens (e.g., <|BGM|>) to corresponding emojis (e.g., 😊, 🎼).
3. Handles sequence cleaning to ensure emojis are correctly placed at the start or end of segments and removes redundant whitespace or specific artifacts like The..

The /transcribe endpoint allows you to upload an audio file for transcription via a POST request using multipart/form-data.

Endpoint Details:

• Path: /transcribe
• Method: POST
• Request Body: multipart/form-data containing a file parameter (the audio file).

Response Schema (200 Success): Returns a JSON object with:

• code (integer): state number
• msg (string): status message
• data (object): The transcription result object.

curl -X 'POST'  
  'http://yourapiaddress/transcribe'  
  -H 'accept: application/json'  
  -H 'Content-Type: multipart/form-data'  
  -F 'file=@path_to_your_audio_file'

Connect to the WebSocket endpoint for real-time transcription.

Endpoint Details:

• Path: /ws/transcribe
• Query Parameters:
  • sv: (Optional) Whether to enable speaker verification. Set to 1 to enable. Default is 0.

Data Formats:

• Upstream (Client to Server): PCM binary data
  • Channels: 1
  • Sample Rate: 16000
  • Bit Depth: 16-bit
• Downstream (Server to Client): JSON String
  • code (integer): state number
  • info (string): meta info
  • data (object): Response object

Client Implementation Example: When using the provided client_wss.html, the connection string follows this pattern:

ws = new WebSocket(`wss://your_wss_server_address/ws/transcribe${sv ? '?sv=1' : ''}`);

The /transcribe endpoint accepts an audio file upload and returns a transcription including emotion and event emojis.

Supported Formats:

• audio/wav
• audio/webm

Request Details:

• Method: POST
• Body: multipart/form-data containing a file field named file.

Response Format: Returns a TranscriptionResponse JSON object:

• code: 0 for success, 1 for error, or the HTTP status code for other errors.
• msg: A status message (e.g., success, transcription time: 1.23 seconds).
• data: The transcribed text with emojis representing emotions (e.g., 😊, 😔) and events (e.g., 🎼, 👏).

import requests

url = "http://localhost:7000/transcribe"
files = {"file": open("audio.wav", "rb")}
response = requests.post(url, files=files)
print(response.json())

The standard JSON response structure used by the transcription API.

{
  "code": 0,
  "msg": "success, transcription time: 0.50 seconds",
  "data": "Hello world 😊"
}

The server is built with FastAPI and can be launched using the server.py script. You can specify the port and optional SSL certificates via command-line arguments.

Arguments:

• --port: The port number to run the server on (default: 7000).
• --certfile: Path to your SSL certificate file.
• --keyfile: Path to your SSL key file.

# Run on default port 7000
python server.py

# Run on a specific port
python server.py --port 8080

# Run with SSL
python server.py --port 443 --certfile cert.pem --keyfile key.pem