repository·main·Indexed 21 days ago
https://github.com/jacktrip/jacktripA multi-machine audio system for high-quality, uncompressed, bidirectional audio streaming over the Internet for network music performance. Supports cross-platform environments (Linux, macOS, Windows, FreeBSD) and includes documentation for hub server containerization via Podman and Docker, AUv2/AUv3 Audio Bridge plugins for DAWs, and integrated libraries such as oscpp for OSC packet handling and Simple-FFT for Fast Fourier Transforms.
JackTrip is a multi-machine audio system designed for network music performance over the Internet. It is cross-platform, supporting Linux, macOS, and Windows.
Key capabilities include:
JackTrip Virtual Studio is an enhanced environment for online musical collaboration. It provides:
The JackTrip Hub Server supports three connection types, each suited for different deployment scenarios:
All types share the same audio packet format and worker pool allocation mechanism.
| Feature | UDP | WebRTC | WebTransport |
|---------|-----|--------|--------------|
| **Transport** | UDP datagrams | WebRTC data channels over UDP | QUIC datagrams over UDP |
| **Signaling** | TCP port 4464 | WebSocket over TCP 4464 | HTTP/3 over UDP 4464 |
| **NAT Traversal** | No | Yes (ICE/STUN/TURN) | Yes (QUIC connection migration) |
| **Browser Support** | No | Yes (all modern browsers) | Yes (Chrome 97+, Edge 97+) |
| **Encryption** | Optional (TLS) | Mandatory (DTLS) | Mandatory (TLS 1.3) |
| **Setup Complexity** | Simple | Complex (ICE negotiation) | Medium (HTTP/3 CONNECT) |
| **Connection Time** | Fastest | Medium (ICE gathering) | Fast (0-RTT after first) |
| **Audio Transport Port** | UDP (61002 + worker_id) | ICE-negotiated UDP ports | UDP 4464 (QUIC) |
| **Library Required** | None (Qt Network) | libdatachannel | msquic |
| **Build Option** | Always available | `-Dlibdatachannel=enabled` | `-Dmsquic=enabled` |When redundancy is enabled with a factor $R$, each UDP datagram contains $R$ full packets concatenated. The newest packet is placed first, followed by older packets in descending order of age.
Sender logic: Prepend the newest full packet to a buffer of previous packets each period. Receiver logic:
SeqNumber.The hub server multiplexes three connection types on a single TCP listen port by inspecting the first three bytes of each new connection.
| First 3 bytes (hex) | Interpretation |
|---|---|
16 03 01 through 16 03 04 | TLS ClientHello (browser wss://) — start TLS handshake; re-detect after decrypt |
| Anything else | Legacy binary hub protocol — read 32-bit LE port number |
Once the TLS handshake is complete, the server inspects the decrypted bytes:
| Decrypted content | Interpretation |
|---|---|
GET /ping … | Health-check endpoint — responds {"status":"OK"} and closes |
GET /webrtc … | HTTP WebSocket upgrade → WebRTC signaling path |
Other GET … | Unsupported path — responds HTTP 404 and closes |
| Other binary data | Authenticated binary hub protocol (credentials follow) |
WebRTC signaling occurs over a WebSocket connection following these steps:
wss://. The server detects the TLS ClientHello via the 3-byte record header.GET /webrtc request with Upgrade: websocket headers.PROTOCOL_DETECT JSON message: {"type": "protocol_detect", "protocol": 2, "clientName": "...", "version": 1}.OFFER (SDP); Server responds with an ANSWER (SDP).ICE_CANDIDATE messages."audio") opens for bidirectional audio datagrams.Signaling Message Format: All signaling messages are JSON objects prefixed with a 4-byte big-endian length prefix over the TCP socket: [4-byte length (BE)] [JSON payload].
JackTrip uses different transport mechanisms depending on the deployment mode:
PacketHeader + raw audio payload. When redundancy is enabled, multiple packets are concatenated into a single datagram for FEC.0xFF used to signal a shutdown ("Peer Stopped").wss://).The library is organized into two primary namespaces representing the two sides of OSC communication:
OSCPP::Client: Used for constructing and building OSC packets. It provides a fluent API to open bundles/messages and write arguments into a provided memory buffer.OSCPP::Server: Used for parsing and reading OSC packets. It provides abstractions like Packet, Bundle, Message, and ArgStream to traverse the structure of received data.Performance Note: The library is designed for real-time sensitive contexts (like audio driver callbacks) because it avoids memory allocation (except when throwing exceptions) and system calls.
The AUv3 plugin uses a hybrid C++/Objective-C architecture to balance high-performance audio processing with macOS framework integration:
JackTripAU: The main AUAudioUnit subclass acting as the Objective-C wrapper for the AUv3 framework.JackTripAUImpl: The C++ implementation class that contains all core audio processing logic.AudioBridgeProcessor: The shared C++ audio processing engine (consistent with AUv2 and VST3 versions) that handles the actual audio exchange.JackTripAUViewController: A native Cocoa-based UI controller for the plugin interface.JackTripAUFactory: Handles the registration of the Audio Unit component.On the wire, the audio payload is planar (non-interleaved) by channel. This means all samples for a single channel are grouped together before moving to the next channel:
Where $N$ is the BufferSize. Note that for mono ($C=1$), the planar and interleaved layouts are identical.
The JackTrip VST3 plugin acts as a bridge between your Digital Audio Workstation (DAW) and a running JackTrip instance on the same machine.