camera-streamer Documentation

camera-streamer is a project focused on providing fully hardware-accelerated streaming of MJPEG and H264 video streams with minimal latency. It is specifically designed to support CSI cameras that provide 10-bit Bayer packed formats by utilizing the dedicated ISP (Image Signal Processor) on Raspberry Pi hardware.

Note: This is a draft project inspired by ustreamer and may not be as feature-complete as other established solutions.

WebRTC support is available via http://<ip>:8080/webrtc.

Requirements:

• An H264 output must be generated by the camera.
• The binary must be compiled with libdatachannel support (enabled by default when running make).

Performance analysis shows that direct ISP-mode generally achieves lower latency compared to libcamera. This is due to how buffers are enqueued and processed. While libcamera is capable of high frame rates (e.g., 120fps), direct ISP-mode provides significantly better processing_ms values by optimizing buffer queuing.

To analyze latency in your own runs, use the -log-filter=buffer_lock flag to see processing_ms and frame_ms in the logs.

To build camera-streamer manually, clone the repository recursively, install the necessary development dependencies, and use make to build and install.

git clone https://github.com/ayufan-research/camera-streamer.git --recursive
apt-get -y install libavformat-dev libavutil-dev libavcodec-dev libcamera-dev liblivemedia-dev v4l-utils pkg-config xxd build-essential cmake libssl-dev

cd camera-streamer/
make
sudo make install

For CSI cameras, you can use a high-performance implementation that directly accesses sensor feeds and passes them via DMA into the bcm2385 ISP module. This mode provides significantly higher frame rates and lower latency than standard specifications.

Important Limitations:

• This mode does not implement automatic brightness control (unlike libcamera).
• It relies on specific sensor parameters (e.g., dumped IMX519 parameters) being fed into the ISP module.
• To support other sensors, the ISP parameters must be adapted accordingly.

# Example usage of the CSI camera script
tools/csi_camera.sh -camera-format=RG10

To support JPEG re-encoding, ensure your /boot/config.txt has sufficient GPU memory (gpu_mem). Recommended values are 160MB or 256MB, though 128MB is a common minimum.

Depending on your camera hardware, add the appropriate dtoverlay settings:

IMX519:

dtoverlay=vc4-kms-v3d,cma-128
gpu_mem=128
dtoverlay=imx519

Arducam 64MP:

gpu_mem=128
dtoverlay=arducam_64mp,media-controller=1

USB Camera:

gpu_mem=128

# Example for IMX519
dtoverlay=vc4-kms-v3d,cma-128
gpu_mem=128 # preferred 160 or 256MB
dtoverlay=imx519

# Example for Arducam 64MP
gpu_mem=128
dtoverlay=arducam_64mp,media-controller=1

# Example for USB cam
gpu_mem=128

To use the Arducam 16MP sensor, you must configure the Raspberry Pi OS via /boot/config.txt and ensure the i2c-dev module is loaded. Note that this setup does not support auto-focus or manual focus through standard drivers due to missing ak7535 support in imx519. Focus must be controlled manually using i2c-tools after the camera has started.

# Add to /boot/config.txt
dtoverlay=imx519,media-controller=0
gpu_mem=160 # at least 128

# Add to /etc/modules-load.d/modules.conf
i2c-dev

You can install camera-streamer using one of two methods:

1. Precompiled Debian Package (Recommended): Download the latest .deb package from the official releases page.
2. Manual Compilation (Advanced): Follow the instructions in docs/install-manual.md to build from source.

To clone the repository including all necessary submodules, use:

git clone --recurse-submodules https://github.com/ayufan/camera-streamer.git

For specialized hardware setups, camera-streamer provides advanced modes:

• CSI High-Performance Mode: Uses the ISP for CSI cameras (see docs/v4l2-isp-mode.md).
• USB High-Performance Mode: Uses direct decoding for USB cameras (see docs/v4l2-usb-mode.md).
• Raspberry Pi High-Compatibility Mode: Uses libcamera on Raspberry Pi (see docs/raspi-libcamera.md).
• Performance Analysis: For details on analyzing performance, see docs/performance-analysis.md.

To discover all available configuration options for the camera, ISP, or codecs, you can use one of two methods:

1. CLI Method: Add the --log-verbose flag to your command.
2. Web Method: Navigate to http://<ip>:8080/option while the streamer is running.

tools/libcamera_camera.sh --camera-list_options ...

The easiest way to run the streamer is by using the provided systemd service templates located in the service/ directory.

1. Enable and start a specific service (e.g., for Arducam 16MP):

   systemctl enable $PWD/service/camera-streamer-arducam-16MP.service
   systemctl start camera-streamer-arducam-16MP

2. Access the stream: Once running, a web server will be available at http://<IP>:8080/.
3. Debugging: If the service fails to start, check the logs using journalctl:

   journalctl -xef -u camera-streamer-arducam-16MP

Configuration for camera-streamer can be managed through several specialized guides:

• Image Settings: To configure resolution, brightness, or image quality, refer to docs/configure.md.
• Streaming Options: For various streaming protocols and modes, refer to docs/streaming.md.
• Example Configurations: You can find ready-to-use example configurations in the service/ directory of the repository.