DOCSEARCHbyAlgolia
Submit Docs

libstreaming

repository·master·Indexed 25 days ago

https://github.com/fyhertz/libstreaming

An Android API for streaming camera and microphone data via RTP over UDP. It supports Android 4.0+ and provides encoders for H.264, H.263, AAC, and AMR. The library offers three signaling methods: RTSP Client, RTSP Server, and SDP Signaling. It utilizes SessionBuilder for configuration and supports multiple encoding methods including MediaRecorder and MediaCodec (Buffer-to-Buffer and Surface-to-Buffer).

Tokens
1.4K
Snippets
3
Records
6
Agent score
36%

What's inside libstreaming

  1. Overview of libstreaming

    master

    libstreaming is an Android API that enables streaming camera and/or microphone data using RTP over UDP. It supports Android 4.0 or more recent and provides encoders for H.264, H.263, AAC, and AMR.

    There are three primary ways to handle 'signaling' (notifying the receiver about incoming streams):

    1. RTSP Client: Best for streaming to a Wowza Media Server.
    2. RTSP Server: The phone acts as a server, waiting for a client to request the stream.
    3. SDP Signaling: Use libstreaming without the RTSP protocol by signaling the session using SDP over any protocol you choose.

  2. Stream H.264 and AAC using SessionBuilder

    master

    Use SessionBuilder to create a Session object. The Session manages one or more Stream objects. You can use the session asynchronously via callbacks or synchronously using sync methods.

    Key Configuration Methods:

    • setCallback(Session.Callback): Register a callback to handle session events (configured, started, stopped, error, bitrate updates).
    • setSurfaceView(SurfaceView): Required for video streaming to provide a valid surface.
    • setContext(Context): Required to allow streams to store/recover data via SharedPreferences.
    • setAudioEncoder(int): Set audio encoding (e.g., SessionBuilder.AUDIO_NONE, SessionBuilder.AUDIO_AAC).
    • setVideoEncoder(int): Set video encoding (e.g., SessionBuilder.VIDEO_H264).
    • setAudioQuality(AudioQuality) and setVideoQuality(VideoQuality): Configure bitrate and resolution.

    Getting the Session Description: Once the session is configured (after onSessionConfigured() is called), use mSession.getSessionDescription() to get an SDP formatted string. This string can be used by receivers like VLC to connect to the stream.

    mSession = SessionBuilder.getInstance()
    .setCallback(this)
    .setSurfaceView(mSurfaceView)
    .setPreviewOrientation(90)
    .setContext(getApplicationContext())
    .setAudioEncoder(SessionBuilder.AUDIO_NONE)
    .setAudioQuality(new AudioQuality(16000, 32000))
    .setVideoEncoder(SessionBuilder.VIDEO_H264)
    .setVideoQuality(new VideoQuality(320,240,20,500000))
    .build();

// In onSessionConfigured callback:
String sdp = mSession.getSessionDescription();
mSession.start();

  3. Set up and use the RTSP Server

    master

    To turn the device into an RTSP server, follow these steps:

    1. Register the Service: Add the RtspServer to your AndroidManifest.xml:
      <service android:name="net.majorkernelpanic.streaming.rtsp.RtspServer"/>
    2. Configure the Port: The port is stored as a String in SharedPreferences. Use RtspServer.KEY_PORT to change it:
      Editor editor = PreferenceManager.getDefaultSharedPreferences(this).edit();
editor.putString(RtspServer.KEY_PORT, "1234");
editor.commit();
    3. Configure the Session: Use SessionBuilder to set up the streams for the server.
    4. Control the Server: Start and stop the server using standard Android Intent service calls.

  4. Configure required Android permissions

    master

    To use libstreaming, you must include the following permissions in your AndroidManifest.xml:

    <uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.CAMERA" />

  5. Use Session in synchronous mode

    master

    If you prefer not to use callbacks, you can use the synchronous methods provided by the Session class. Note that these methods block the calling thread until the operation completes.

    // Blocks until the all streams are configured
try {
     mSession.syncConfigure();
} catch (Exception e) {
     ...
}
String sdp = mSession.getSessionDescription();

// Blocks until streaming actually starts.
try {
     mSession.syncStart();
} catch (Exception e) {
     ...
}

mSession.syncStop();

  6. Reference: Encoding Methods

    master

    libstreaming supports three ways to retrieve encoded data from Android peripherals:

    1. MediaRecorder API: Uses a ParcelFileDescriptor (on newer Android versions) to write to a socket. It is a fallback method but may have jitter and lip-sync issues.
    2. MediaCodec API (Buffer-to-Buffer): Uses dequeueInputBuffer and queueInputBuffer. Requires Android 4.1+. The hw package helps manage various color formats.
    3. MediaCodec API (Surface-to-Buffer): Uses createInputSurface(). This is the best way to encode raw video but requires Android 4.3+. The gl package handles this method.