DOCSEARCHbyAlgolia
Submit Docs

HaishinKit Documentation

repository·main·Indexed 25 days ago

https://github.com/haishinkit/haishinkit.swift

A camera and microphone streaming library for Apple platforms (iOS, macOS, tvOS, visionOS) supporting RTMP, SRT, and WHEP/WHIP protocols. It provides modules for live mixing via MediaMixer, video overlays using ImageScreenObject and VideoTrackScreenObject, and protocol-specific stacks including RTMPHaishinKit, SRTHaishinKit, RTCHaishinKit, and MoQTHaishinKit.

Tokens
5.6K
Snippets
24
Records
34
Agent score
85%

What's inside HaishinKit

  3. Overview of HaishinKit Modules

    main

    HaishinKit provides camera and microphone mixing functionality for live streaming. The library is divided into several modules based on the protocol stack required:

    • HaishinKit: The core module providing mixing and common processing.
    • RTMPHaishinKit: Provides the RTMP protocol stack.
    • SRTHaishinKit: Provides the SRT protocol stack.
    • RTCHaishinKit: Provides the WebRTC WHEP/WHIP protocol stack (currently in alpha).
    • MoQTHaishinKit: Provides the MoQT protocol stack (currently in alpha).

  4. Understand Enhanced RTMP support in RTMPHaishinKit

    main

    RTMPHaishinKit supports features of the Enhanced RTMP standard developed by Veovera Software Organization.

    Important Requirements:

    • Server Support: Enhanced RTMP requires the streaming server to support the specific Enhanced RTMP version and features you intend to use. Always verify your server's compatibility before implementation.
    • Hardware Compatibility: Support for certain codecs (like AV1) is dependent on hardware compatibility and is planned for future implementation.

  8. Use the StreamSession API for RTMP and SRT

    main

    The StreamSession API provides a unified interface for implementing clients using RTMP and SRT protocols. It includes internal retry handling.

    1. Preparation

    You must first register the session factories with the StreamSessionBuilderFactory before creating sessions.

    2. Creating a Session

    • RTMP: Provide a URL combining the RTMP connection URL and the streamName.
    • SRT: Provide the SRT connection URL.

    3. Connecting

    Use the connect method to initiate publishing or playback. The method accepts a completion handler for disconnection events.

  9. Playback RTMP streams with RTMPHaishinKit

    main

    To play an RTMP stream, initialize an RTMPConnection and an RTMPStream. You must attach an AudioPlayer (using AVAudioEngine) to the stream to hear audio, and an MTHKView to the stream to see video.

    let connection = RTMPConnection()
let stream = RTMPStream(connection: connection)
let audioPlayer = AudioPlayer(AVAudioEngine())

let hkView = MTHKView(frame: view.bounds)

Task { MainActor in
  await stream.addOutput(hkView)
}

Task {
  // requires attachAudioPlayer
  await stream.attachAudioPlayer(audioPlayer)

  do {
    try await connection.connect("rtmp://localhost/appName/instanceName")
    try await stream.play(streamName)
  } catch RTMPConnection.Error.requestFailed(let response) {
    print(response)
  } catch RTMPStream.Error.requestFailed(let response) {
    print(response)
  } catch {
    print(error)
  }
}

  10. Publish RTMP streams with RTMPHaishinKit

    main

    To publish a stream, you need to set up a MediaMixer to capture audio/video, an RTMPConnection to establish the network link, and an RTMPStream to handle the data flow. You can use MTHKView to preview the stream locally.

    Supported codecs:

    • H264, HEVC, AAC, and OPUS for publishing.
    • H264, HEVC, and AAC for playback.
    let mixer = MediaMixer()
let connection = RTMPConnection()
let stream = RTMPStream(connection: connection)
let hkView = MTHKView(frame: view.bounds)

Task {
  do {
    try await mixer.attachAudio(AVCaptureDevice.default(for: .audio))
  } catch {
    print(error)
  }

  do {
    try await mixer.attachVideo(AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back))
  } catch {
    print(error)
  }

  await mixer.addOutput(stream)
}

Task { MainActor in
  await stream.addOutput(hkView)
  // add ViewController#view
  view.addSubview(hkView)
}

Task {
  do {
    try await connection.connect("rtmp://localhost/appName/instanceName")
    try await stream.publish(streamName)
  } catch RTMPConnection.Error.requestFailed(let response) {
    print(response)
  } catch RTMPStream.Error.requestFailed(let response) {
    print(response)
  } catch {
    print(error)
  }
}

  11. Configure Info.plist permissions for Camera and Microphone

    main

    To access the camera and microphone on iOS/macOS, you must include the following keys in your Info.plist file with appropriate usage descriptions.

    <key>NSCameraUsageDescription</key>
<string>your usage description here</string>
<key>NSMicrophoneUsageDescription</key>
<string>your usage description here</string>