DOCSEARCHbyAlgolia
Submit Docs

Membrane Framework Core

repository·master·Indexed 23 days ago

https://github.com/membraneframework/membrane_core

The central multimedia streaming and processing framework written in Elixir. membrane_core provides the primary API for creating pipelines and custom elements, allowing developers to build media servers capable of transcoding, mixing, and streaming via protocols such as WebRTC, RTSP, and RTMP. It serves as the foundation for a modular ecosystem of plugins, formats, and standalone libraries.

Tokens
36.3K
Snippets
66
Records
258
Agent score
78%

What's inside membrane_core

  1. Overview of Membrane Framework core packages

    master

    The Membrane Framework ecosystem consists of several core packages for multimedia processing:

    • membrane_sdk: Provides the full power of Membrane in a single package.
    • membrane_core: The central multimedia processing framework written in Elixir.
    • membrane_rtc_engine: A customizable Real-time Communication Engine/SFU library focused on WebRTC.
    • kino_membrane: Utilities for introspecting Membrane pipelines within Livebook.
    • docker_membrane: [Labs] A Docker image (Ubuntu-based) containing Erlang, Elixir, and necessary libraries for running and testing Membrane.
    • membrane_demo: Practical examples of using the framework.
    • membrane_tutorials: Assets and text used in official tutorials.
    • boombox: A simple streaming tool built on top of Membrane.

  6. Explore Membrane Framework plugins and formats

    master

    The Membrane Framework ecosystem is composed of various specialized packages categorized into plugins, formats, standalone media libraries, and utilities. Use these packages to extend your media pipelines with support for specific cloud services, media containers, codecs, or protocols.

    Plugins

    Plugins provide integration with external services or specific media handling capabilities. Examples include:

    • membrane_aws_plugin: AWS integration.
    • membrane_agora_plugin: Sink for Agora Server Gateway.
    • membrane_element_gcloud_speech_to_text: Google Cloud Speech-to-Text API.
    • membrane_element_ibm_speech_to_text: IBM Cloud Speech-to-Text service.
    • membrane_s3_plugin: S3 sources and destinations.
    • membrane_transcription: Prototype transcription.

    Formats

    Format packages define how different media containers and codecs are represented within the framework. Examples include:

    • Containers: membrane_rtp_format, membrane_cmaf_format, membrane_matroska_format, membrane_mp4_format.
    • Audio Codecs: membrane_raw_audio_format, membrane_aac_format, membrane_opus_format, membrane_flac_format, membrane_mpegaudio_format, membrane_g711_format.
    • Video Codecs: membrane_raw_video_format, membrane_h264_format, membrane_vp8_format, membrane_vp9_format, membrane_av1_format, membrane_h265_format.

    Standalone Media Libraries

    These are specialized Elixir libraries for handling specific media protocols or standards, often used alongside the framework:

    • ex_webrtc: W3C WebRTC API implementation.
    • ex_sdp: Session Description Protocol parser/serializer.
    • ex_libnice: ICE protocol support.
    • ex_libsrtp: libsrtp bindings.
    • ex_m3u8: M3U8 manifest serialization/deserialization.
    • ex_hls: HLS stream handling.
    • ex_libsrt: libsrt client/server API bindings.
    • membrane_rtsp: RTSP client.
    • membrane_ffmpeg_generator: FFmpeg generator for tests and demos.

  11. Explore Membrane Framework packages

    master

    The Membrane Framework ecosystem consists of several core packages, specialized plugins, and utility tools.

    Core Packages

    • membrane_sdk: Provides the full power of Membrane in a single package.
    • membrane_core: The central multimedia processing framework written in Elixir.
    • membrane_rtc_engine: A customizable Real-time Communication Engine/SFU library focused on WebRTC.
    • kino_membrane: Utilities for introspecting Membrane pipelines within Livebook.
    • boombox: A simple streaming tool built on top of Membrane.

    General Purpose Plugins

    • membrane_file_plugin: For reading and writing to files.
    • membrane_hackney_plugin: HTTP sink and source based on Hackney.
    • membrane_scissors_plugin: An element for cutting off parts of a stream.
    • membrane_tee_plugin: Splits data from a single input to multiple outputs.
    • membrane_funnel_plugin: Merges multiple input streams into a single output.
    • membrane_realtimer_plugin: Limits playback speed to realtime based on buffer timestamps.
    • membrane_stream_plugin: Records entire streams through Membrane pads into a binary format for later replay.
    • membrane_fake_plugin: Fake sinks that drop incoming data.
    • membrane_transcoder_plugin: Provides audio and video transcoding capabilities.
    • membrane_generator_plugin: Generates video and audio samples.
    • membrane_live_framerate_converter_plugin: A filter that drops or duplicates frames to match a target framerate (designed for realtime applications).
    • membrane_template_plugin: Provides templates for Membrane Elements.
    • membrane_pcap_plugin [Labs]: A PCAP source for reading captured packets in pcap format.

    AI Plugins

    • membrane_whisper_plugin: Integrates OpenAI's Whisper into audio processing pipelines.
    • membrane_yolo_plugin: Applies YOLO object detection on raw video frames.