Amazon Kinesis Video Streams WebRTC SDK for C

repository·main·Indexed 22 days ago

https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c

A pure C implementation of a WebRTC client designed for Amazon Kinesis Video Streams. It provides a small-footprint, portable implementation for embedded devices and various platforms, supporting VP8, H264, Opus, and G.711 PCM codecs. The SDK includes a signaling client for cross-platform connectivity with Android, iOS, and Web SDKs, and supports DataChannels, STUN/TURN, and media ingestion into Kinesis Video Streams.

Tokens
12.6K
Snippets
21
Records
54
Agent score
78%

What's inside Amazon Kinesis Video Streams WebRTC SDK for C

  1. Overview of Amazon Kinesis Video Streams C WebRTC SDK

    main

    The Amazon Kinesis Video Streams C WebRTC SDK is a pure C WebRTC client designed for Amazon Kinesis Video Streams. It provides a portable, small-footprint implementation suitable for embedded devices and various platforms.

    Key Capabilities

    • Media Support: Supports VP8, H264, Opus, and G.711 PCM (A-law and µ-law).
    • Media Pipeline Control: Provides raw media for input/output and callbacks for Congestion Control, FIR, and PLI (via RtcRtpTransceiver).
    • Networking & Connectivity: Includes support for DataChannels, NACKs, STUN/TURN, and both IPv4/IPv6.
    • Signaling: Includes a Signaling Client that connects to the Kinesis Video Streams signaling backend, enabling interoperability with Android, iOS, and Web SDKs.
    • Storage: Supports ingesting media into a Kinesis Video Stream.

    Portability and Size

    • Platforms: Tested on Linux and MacOS; supports x64 and ARMv5 architectures.
    • Footprint: Small install size (sub 200k library size).
    • Dependencies: Relies on OpenSSL, libsrtp, libjsmn, libusrsctp, and libwebsockets.
  2. What is Transport-Wide Congestion Control (TWCC)?

    main

    Transport-Wide Congestion Control (TWCC) is a mechanism used to prevent video quality degradation and network stalls during congestion. It creates a feedback loop where the receiver reports when specific packets arrive, allowing the sender to detect growing delays and adjust the video/audio bitrate accordingly.

    How it works:

    1. Sender: Numbers each outgoing packet sequentially.
    2. Receiver: Notes arrival times for these numbered packets and periodically sends feedback.
    3. Sender: Compares send times vs. receive times to detect congestion via increasing delays.
    4. Sender: Adjusts the encoder bitrate up or down based on this analysis.
  3. How the AIMD Bitrate Controller works

    main

    The SDK provides an AIMD (Additive Increase, Multiplicative Decrease) controller, demonstrated in the sampleOnPeerCongestionFeedback sample. It manages bitrate by reacting to two independent signals: Packet Loss and Delay Trend.

    Decision Logic

    The controller reacts to whichever signal indicates worse conditions.

    SignalMeasurementCongested ThresholdClear Threshold
    Packet loss(tx_packets - rx_packets) / tx_packets (EMA smoothed)> 5%<= 2%
    Delay trendOutput of computeTwccTrendline> 0.5 ms< -0.1 ms

    Bitrate Adjustments

    • Congestion (Multiplicative Decrease): When congestion is detected, the bitrate is cut by a factor based on severity. If both signals indicate congestion, the minimum (most aggressive) factor is used.
    • Clear (Additive Increase): When conditions are clear, the bitrate is increased in small steps. Adjustments are rate-limited by TWCC_BITRATE_ADJUSTMENT_INTERVAL_MS to prevent oscillation.

    Bitrate Bounds (Clamping)

    To prevent the encoder from receiving extreme values, the resulting bitrate is always clamped to a [min, max] range using the following logic:

    videoBitrate = MAX(MIN(videoBitrate, MAX_VIDEO_BITRATE_KBPS), MIN_VIDEO_BITRATE_KBPS);
    audioBitrate = MAX(MIN(audioBitrate, MAX_AUDIO_BITRATE_BPS), MIN_AUDIO_BITRATE_BPS);
  4. How the built-in Trendline Estimator detects congestion

    main

    The SDK includes a default estimator, computeTwccTrendline, which detects network congestion by measuring increasing delays in packet arrival relative to their send times.

    It uses an algorithm based on Ordinary Least Squares (OLS) regression and Exponential Moving Average (EMA) smoothing to calculate a delayTrend in milliseconds.

    The logic follows these steps:

    1. Inter-packet delay variation: Calculates delay_variation = (recv_gap) - (send_gap) for consecutive acknowledged packets.
    2. Accumulated delay: Maintains a running sum of these variations to track queuing buildup.
    3. Linear regression (OLS): Fits a line to the accumulated delay. The slope indicates the state:
      • Positive slope: Congestion (delay is growing).
      • Zero slope: Stable network.
      • Negative slope: Recovery (delay is shrinking).
    4. EMA smoothing: Applies smoothed = alpha * raw_slope + (1 - alpha) * previous_smoothed to filter out jitter and react only to sustained trends.
    5. Output: A single delayTrend value used to drive bandwidth decisions.
  5. Use pre-generated certificates to reduce connection latency

    main

    Generating certificates via createDtlsSession() can take 5-15 seconds on low-performance embedded devices. To avoid this latency during peer connection creation, you can pre-generate certificates and dequeue them from a pool.

    Best Practice: Rotate certificates often (preferably for every peer connection) to maintain security.

    Configuration: Use the following environment variables to control pre-generation behavior:

    • KVS_PRE_GENERATE_CERT_ENABLED: Enable/disable (TRUE/FALSE).
    • KVS_PRE_GENERATE_CERT_PERIOD_MS: Interval to check/generate (100-60000 ms).
    • KVS_PRE_GENERATE_CERT_MAX: Max certificates to keep in queue (0-10).
    retStatus = stackQueueDequeue(pSampleConfiguration->pregeneratedCertificates, &data);
    if (retStatus == STATUS_SUCCESS) {
        pRtcCertificate = (PRtcCertificate) data;
        configuration.certificates[0] = *pRtcCertificate;
    }
  6. Configure ICE related timeouts

    main

    You can adjust ICE timeout values in KvsRtcConfiguration to handle poor network conditions.

    Key parameters include:

    • iceCandidateNominationTimeout: Increase this if you need more time to gather relay (TURN) candidates before nomination begins.
    • iceLocalCandidateGatheringTimeout: Increase this to allow more time for gathering all potential candidates (host, srflx, relay) on slow networks.
    • iceConnectionCheckTimeout: Increase this for unstable/slow networks to allow more time for binding request/response exchanges.
    • iceConnectionCheckPollingInterval: Controls the frequency of connectivity checks. While decreasing this can speed up connection in high-performance networks, it is generally recommended to stick to the default (50 ms) to remain compliant with RFC 8445.
  7. Configure AWS credentials and region

    main

    Before running the SDK or its samples, you must set up your environment with your AWS account credentials and the target AWS region. If you are using temporary credentials, you must also set the AWS_SESSION_TOKEN.

    To connect to Kinesis Video Streams (KVS), set the following environment variables:

    export AWS_ACCESS_KEY_ID=<AWS account access key>
    export AWS_SECRET_ACCESS_KEY=<AWS account secret key>
    # Optional: for temporary credentials
    export AWS_SESSION_TOKEN=<session token>
    # Optional: defaults to us-west-2 if not set
    export AWS_DEFAULT_REGION=<AWS region>
    export AWS_ACCESS_KEY_ID=<AWS account access key>
    export AWS_SECRET_ACCESS_KEY=<AWS account secret key>
  8. Enable TWCC in the SDK

    main

    To use TWCC, you must enable it in your configuration. In the provided GStreamer samples, TWCC is enabled by default.

    To ensure TWCC is active, ensure the following configuration settings are applied:

    • pSampleConfiguration->enableTwcc = TRUE
    • KvsRtcConfiguration.disableSenderSideBandwidthEstimation = FALSE

    Note: TWCC bitrate adaptation is only implemented in GStreamer-based samples. Non-GStreamer samples (like kvsWebrtcClientMaster) do not include encoder bitrate control because they send pre-encoded frames from a file rather than a live stream.

    // TWCC is enabled by default in samples:
    pSampleConfiguration->enableTwcc = TRUE;
    
    // Ensure sender-side bandwidth estimation is NOT disabled:
    KvsRtcConfiguration.disableSenderSideBandwidthEstimation = FALSE;
  9. Run Peer-to-Peer WebRTC samples

    main

    Peer-to-peer samples allow you to simulate a Master (sender) and a Viewer (receiver) connection.

    kvsWebrtcClientMaster

    Sends sample H264/Opus frames via WebRTC. It can also accept incoming audio.

    Usage:

    ./samples/kvsWebrtcClientMaster <channelName> <audio-codec> <video-codec>
    • audio-codec: opus (default)
    • video-codec: h264 (default), h265

    kvsWebrtcClientMasterGstSample

    Sends media using a GStreamer pipeline (test sources, device sources, or RTSP).

    Usage:

    ./samples/kvsWebrtcClientMasterGstSample <channelName> <mediaType> <sourceType> [codec_args]
    • mediaType: audio-video, video-only
    • sourceType: testsrc, devicesrc, rtspsrc
    • rtspsrc usage: ./samples/kvsWebrtcClientMasterGstSample <channelName> <mediaType> rtspsrc rtsp://<rtspUri>

    Example (testsrc with codecs):

    ./samples/kvsWebrtcClientMasterGstSample <channelName> audio-video testsrc opus h264

    kvsWebrtcClientViewer

    Accepts sample H264/Opus frames and logs buffer sizes.

    Usage:

    ./samples/kvsWebrtcClientViewer <channelName> <audio-codec> <video-codec>

    kvsWebrtcClientViewerGstSample

    Similar to the standard viewer, but uses GStreamer filesink to save received media to a file.

    Usage:

    ./samples/kvsWebrtcClientViewerGstSample <channelName> <mediaType> <audio-codec> <video-codec>
  10. Use the Profile Statistics Aggregator to process logs

    main

    The aggregate.py script aggregates PROFILE log metrics from Kinesis Video Streams WebRTC logs. You can provide a single file, multiple specific files, or use wildcards to target all logs in a directory.

    Supported usage patterns:

    • Single file: Pass the path to one log file.
    • Multiple files: Pass multiple paths as arguments.
    • Directory pattern: Use shell wildcards (e.g., *) to include all matching log files in a directory.
    # Single file
    python3 ./scripts/profile-stats/aggregate.py /path/to/log.txt
    
    # Multiple files
    python3 ./scripts/profile-stats/aggregate.py /path/to/log1.txt /path/to/log2.txt
    
    # All logs in directory
    python3 ./scripts/profile-stats/aggregate.py ./build/kvsFileLogFilter.*
  11. Build with system dependencies instead of source

    main

    If you want to use libraries already installed on your system (e.g., via apt or brew) instead of having the SDK build them from source, use -DBUILD_DEPENDENCIES=OFF.

    Required Versions:

    • libmbedtls: >= 2.25.0 & < 4.x.x
    • libopenssl: = 1.1.1x
    • libsrtp2: <= 2.5.0
    • libusrsctp: <= 0.9.5.0
    • libwebsockets: >= 4.2.0

    Usage Examples:

    To use OpenSSL:

    cmake .. -DBUILD_DEPENDENCIES=OFF -DUSE_OPENSSL=ON

    To use MBedTLS:

    cmake .. -DBUILD_DEPENDENCIES=OFF -DUSE_OPENSSL=OFF -DUSE_MBEDTLS=ON
    CAUTION

    System-installed libwebsockets and libsrtp are typically built against OpenSSL. If you use -DUSE_MBEDTLS=ON with system packages, you may encounter linker/runtime errors. To ensure compatibility with mbedTLS, it is recommended to build these dependencies from source (-DBUILD_DEPENDENCIES=ON).

  12. Profile high memory or CPU usage with gperftools

    main

    To identify code paths causing high resource usage, recompile the SDK with the LINK_PROFILER flag:

    cmake .. -DLINK_PROFILER=ON

    This links the SDK with gperftools. Once compiled, use the following environment variables to generate profiles:

    Heap Profile:

    HEAPPROFILE=/tmp/heap.prof /path/to/your/binary

    CPU Profile:

    CPUPROFILE=/tmp/cpu.prof /path/to/your/binary