rtsp-client-android

Latency in this library is split into two categories: Network and Video Decoder latency.

Reducing Network Latency

1. Use Ethernet: Connect both the Android device and the RTSP camera via Ethernet rather than WiFi.
2. Lower Bitrate: Decrease the stream bitrate on the RTSP camera. Smaller frames transfer faster.

Reducing Video Decoder Latency

Decoder latency varies by device and stream (ranging from 20ms to 1200ms). To minimize it:

1. Use Baseline Profile: Use the lowest possible H.264 video stream profile and level. The Baseline profile typically results in the lowest latency. You can check profile_idc and level_idc in the logs by enabling debug mode in the library.
2. Minimize Reorder Frames: Ensure the max_num_reorder_frames parameter in the stream is set to 0.
3. Experimental Feature: Use the experimentalUpdateSpsFrameWithLowLatencyParams feature to rewrite the configuration frame at runtime with low-latency parameters.

The easiest way to show a video stream is to use the provided UI components.

• RtspSurfaceView (Recommended): Best performance and lowest battery usage. Use this if you just need to display the video. To capture bitmaps from this view, use PixelCopy.request.
• RtspImageView: Use this if you need to frequently extract bitmaps for further processing (e.g., for AI/Computer Vision), as it offers better performance for bitmap retrieval than PixelCopy on a SurfaceView.

XML Layout Setup

Add either component to your layout file:

<com.alexvas.rtsp.widget.RtspSurfaceView
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/svVideo" />

<com.alexvas.rtsp.widget.RtspImageView
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/ivVideo" />

Kotlin Implementation

Initialize and start the stream using the URI, username, and password:

val uri = Uri.parse("rtsps://10.0.1.3/test.sdp")
val username = "admin"
val password = "secret"

// For RtspSurfaceView
svVideo.init(uri, username, password)
svVideo.start(
    requestVideo = true,
    requestAudio = true,
    requestApplication = false
)

// ... later
svVideo.stop()

val uri = Uri.parse("rtsps://10.0.1.3/test.sdp")
val username = "admin"
val password = "secret"
svVideo.init(uri, username, password)
svVideo.start(
    requestVideo = true,
    requestAudio = true,
    requestApplication = false)
// ...
svVideo.stop()

RtspImageView is a custom ImageView designed for low-latency RTSP stream playback using bitmaps. It manages the underlying RtspProcessor and video decoding lifecycle automatically.

To use it, follow these steps:

1. Initialize the view with an RTSP URI and optional credentials using init().
2. Start the stream using start() by specifying which tracks (video, audio, application) to request.
3. Use stop() to terminate the stream.

You can also listen for individual bitmap frames via onRtspImageBitmapListener or monitor stream status and data via setStatusListener and setDataListener.

// 1. Initialize the view (e.g., in an Activity or Fragment)
val rtspImageView = findViewById<RtspImageView>(R.id.rtsp_image_view)
rtspImageView.init(
    uri = Uri.parse("rtsp://your_stream_url"),
    username = "user",
    password = "pass"
)

// 2. Optional: Listen for raw bitmaps
rtspImageView.onRtspImageBitmapListener = object : RtspImageView.RtspImageBitmapListener {
    override fun onRtspImageBitmapObtained(bitmap: Bitmap) {
        // Handle the raw bitmap frame
    }
}

// 3. Start playback
rtspImageView.start(requestVideo = true, requestAudio = false, requestApplication = false)

// 4. Stop playback when done
// rtspImageView.stop()

If you need to obtain raw frames (e.g., for writing to an MP4 via a muxer) without using the built-in UI components, use the RtspClient API. You must implement RtspClient.RtspClientListener to handle incoming NAL units, audio samples, or application data.

Implementation Example

val rtspClientListener = object: RtspClient.RtspClientListener {
    override fun onRtspConnecting() {}
    override fun onRtspConnected(sdpInfo: SdpInfo) {}
    override fun onRtspVideoNalUnitReceived(data: ByteArray, offset: Int, length: Int, timestamp: Long) {
        // Send raw H264/H265 NAL unit to decoder
    }
    override fun onRtspAudioSampleReceived(data: ByteArray, offset: Int, length: Int, timestamp: Long) {
        // Send raw audio to decoder
    }
    override fun onRtspApplicationDataReceived(data: ByteArray, offset: Int, length: Int, timestamp: Long) {
        // Send raw application data to app specific parser
    }
    override fun onRtspDisconnected() {}
    override fun onRtspFailedUnauthorized() {
        Log.e(TAG, "RTSP failed unauthorized")
    }
    override fun onRtspFailed(message: String?) {
        Log.e(TAG, "RTSP failed with message '$message'")
    }
}

val uri = Uri.parse("rtsps://10.0.1.3/test.sdp")
val username = "admin"
val password = "secret"
val stopped = new AtomicBoolean(false)

// Create a socket (e.g., via NetUtils for SSL/TLS)
val sslSocket = NetUtils.createSslSocketAndConnect(uri.getHost(), uri.getPort(), 5000)

val rtspClient = RtspClient.Builder(sslSocket, uri.toString(), stopped, rtspClientListener)
    .requestVideo(true)
    .requestAudio(true)
    .withDebug(false)
    .withUserAgent("RTSP client")
    .withCredentials(username, password)
    .build()

// Blocking call until 'stopped' becomes true or connection fails
rtspClient.execute()

NetUtils.closeSocket(sslSocket)