StreamPack Android SDK

The TSMuxer is a minimalist MPEG-TS muxer designed to manage packet encapsulation and section tables (PAT, PMT, SDT). It allows you to manage multiple services and streams, and supports encoding AAC and H264 frames into the transport stream.

Key capabilities include:

• Adding and removing streams.
• Adding and removing services.
• Encoding AAC and H264 frames.

When deciding how to handle simultaneous streaming and recording, choose based on your hardware constraints and quality requirements:

| Feature | DualEndpoint | DualStreamer | | :--- | :--- | : | | Encoder Strategy | Single encoder (1 video, 1 audio) | Multi-encoder (independent) | | Resource Usage | Low (Efficient) | High | | Flexibility | Same settings for both outputs | Different settings (bitrate, codec, etc.) per output | | Best Use Case | When hardware resources are limited | | Best Use Case | When you need high-quality recording separate from live stream settings |

A Streamer is the primary class used to stream audio and video from a source to an endpoint. It manages audio/video sources, encoders, and the endpoint. Depending on your requirements for independent outputs, you should choose one of the following types:

• SingleStreamer: Use this for a single output (e.g., streaming live OR recording to a file).
• DualStreamer: Use this for exactly two independent outputs (e.g., streaming live AND recording to a file simultaneously).
• StreamerPipeline: Use this for complex scenarios requiring multiple independent outputs (e.g., saving audio to one file and video to another file).

To ensure the stream orientation matches the device, use a RotationProvider. StreamPack provides two built-in implementations:

• SensorRotationProvider: Uses OrientationEventListener to follow physical device orientation.
• DisplayRotationProvider: Uses DisplayManager and returns the last known orientation if the display is locked.

Usage via Listener

val rotationProvider = SensorRotationProvider(context)
rotationProvider.addListener(object : IRotationProvider.Listener {
    override fun onOrientationChanged(rotation: Int) {
        streamer.setTargetRotation(rotation)
    }
})

Usage via Coroutines (Flow)

Transform a provider into a Flow using asFlowProvider():

val rotationFlowProvider = rotationProvider.asFlowProvider()
rotationFlowProvider.rotationFlow.collect { rotation ->
    streamer.setTargetRotation(rotation)
}

The SingleStreamer is a specialized Streamer that targets a single output. Several pre-defined implementations and factories are available:

• AudioOnlySingleStreamer: Streams only from an audio source (defaults to microphone).
• VideoOnlySingleStreamer: Streams only from a video source (defaults to microphone).
• cameraSingleStreamer: A factory to create a streamer using a camera source.
• videoMediaProjectionSingleStreamer: A factory for media projection video sources (requires setting activity result).
• audioVideoMediaProjectionSingleStreamer: A factory for media projection video and audio sources (requires setting activity result).

By default, these use a DynamicEndpoint, which automatically infers the protocol from the MediaDescriptor provided during open or startStream calls.

A Streamer represents a complete streaming pipeline from capture to endpoint (encoding, muxing, and sending). StreamPack provides different types based on your output requirements:

• SingleStreamer: Best for a single output (e.g., just live streaming or just recording).
• DualStreamer: Provides two independent outputs (e.g., independent live stream and independent local recording).
• StreamerPipeline: For complex scenarios requiring multiple independent outputs (e.g., sending video to a server while saving audio to a separate file).

Streamers can be instantiated via factories (like cameraSingleStreamer) or manually by setting setAudioSource and setVideoSource using SourceFactory implementations.

For scenarios where you need different settings (such as different codecs or bitrates) for your live stream and your recording, use a DualStreamer.

Unlike DualEndpoint, which uses a single encoder for both outputs, DualStreamer manages two independent outputs. This provides maximum flexibility at the cost of higher hardware resource consumption because it uses multiple encoders.

val streamer = DualStreamer(context)

// To start the live stream on the first output
streamer.first.startStream("rtmp://serverip:1935/s/streamKey")
// To start the recording on the second output
streamer.second.startStream("file:///path.to.file.mp4")

To use the TSMuxer, follow these three steps:

1. Instantiate the muxer: Provide an IMuxerListerner to handle the output packets.
2. Register services and streams: Use addStreams to register service information and audio/video configurations. This returns a map where you can retrieve the streamPid using your configuration object as a key.
3. Encode frames: Pass encoded frames and their corresponding streamPid to the encode method.

// 1. Instantiate
val muxer = TSMuxer(listener = object : IMuxerListerner {
    override fun onOutputFrame(packet: Packet) {
        // Use packet
    }
})

// 2. Register services and streams
val streamPid = muxer.addStreams(serviceInfo, listOf(audioConfig))[audioConfig]

// 3. Encode frames
muxer.encode(frame, streamPid)