nokhwa

The nokhwa-core crate provides the fundamental types and abstractions for camera interaction, video streaming, and frame processing. The public API is organized into several specialized modules:

• camera: High-level camera management and device discovery.
• codec: Video codec definitions and handling.
• control: Camera control interfaces (e.g., brightness, focus).
• decoder: Logic for decoding video streams.
• error: Error types for the library.
• format_request: Structures for requesting specific frame formats.
• frame_buffer: Management of raw frame data buffers.
• frame_format: Definitions of pixel formats and frame layouts.
• image: Image processing and representation utilities.
• metadata: Camera and frame metadata.
• pixel_destination: Logic for directing pixel data to specific outputs.
• platform: Platform-specific abstraction layers.
• ranges: Utility for handling ranges (e.g., supported resolutions).
• stream: Stream management and lifecycle.
• traits: Core traits for implementing custom decoders or camera backends.
• types: Common primitive types used across the library.
• utils: General utility functions.

The primary way to interact with webcams in Nokhwa is through the Camera struct. To use the library, you must enable at least one input-* feature (the recommended default is input-native).

Commonly used types and modules include:

• Camera: The main struct for capturing video.
• init: Provides initialization logic (via pub use init::*).
• query: Provides methods for querying available devices and capabilities.
• nokhwa_core::frame_buffer::FrameBuffer: Used for handling captured frames.
• NokhwaError: The standard error type for the library.

Depending on your requirements, you can also use specialized camera types:

• CallbackCamera: (Requires output-threaded feature) A camera that runs in a separate thread and uses callbacks to deliver frames.
• async_camera: (Requires output-async feature) Provides asynchronous camera capabilities.

use nokhwa::prelude::*;nokhwa::Camera;

// Example conceptual usage:
// let camera = Camera::new(index, ...)?;
// let frame = camera.get_frame()?;

The AVFoundationCaptureDevice struct provides a macOS-specific implementation of the CaptureTrait using Apple's AVFoundation framework.

Important Requirements & Quirks

• Initialization: You must call nokhwa_initialize before performing any operations with AVFoundation.
• Platform: This backend only works on 64-bit macOS platforms.
• Permissions: If the application has not been granted camera permissions by the user, calling new() or open_stream() will result in an error.
• Limitations:
  • FPS (Frames Per Second) adjustment is not supported.
  • While iOS is technically allowed by the underlying framework, it is not officially supported and may not function correctly.

Lifecycle

1. Create: Use new(index, req_fmt) to instantiate the device.
2. Open: Call open_stream() to start the AVCaptureSession.
3. Capture: Use frame() to retrieve the next FrameBuffer.
4. Stop: Call stop_stream() to release the session and inputs, or allow the object to be dropped.

use nokhwa_core::types::{CameraIndex, RequestedFormat, RequestedFormatType};
use nokhwa_core::pixel_format::RgbFormat;

// Note: Ensure nokhwa_initialize() is called elsewhere in your setup
let index = CameraIndex::Index(0);
let req_fmt = RequestedFormat::new::<RgbFormat>(RequestedFormatType::Exact(camera_format));
let mut device = AVFoundationCaptureDevice::new(&index, req_fmt)?;

When selecting a camera format from a list of available options, you can use FormatRequestType to define a strategy for picking the best match. This is an optional helper; alternatively, you can use crate::camera::Camera::enumerate_formats for a simpler approach.

Available strategies:

• Closest: Attempts to find the CameraFormat closest to a preferred resolution and frame rate. It can optionally take Range constraints for resolution and frame_rate to filter valid candidates.
• HighestFrameRate: Filters formats within a specific frame_rate range and sorts them to prioritize the highest available rate.
• HighestResolution: Filters formats within a specific resolution range and sorts them to prioritize the highest available resolution.
• Exact: Returns only formats that exactly match the provided resolution and frame_rate.
• Any: Returns all available formats without filtering or sorting.

use nokhwa::format_request::FormatRequestType;
use nokhwa::types::{Resolution, FrameRate};
use nokhwa::ranges::Range;

// Example: Requesting the closest format to a specific target
let strategy = FormatRequestType::Closest {
    resolution: Some(Range::new(Resolution::new(1280, 720), Resolution::new(1920, 1080))),
    preferred_resolution: Some(Resolution::new(1920, 1080)),
    frame_rate: None,
    preferred_frame_rate: Some(FrameRate::from_fps(30.0)),
};

The GStreamerCaptureDevice provides a way to capture video using GStreamer pipelines.

Warning: This backend is deprecated since version 0.10. It is recommended to use native backends (V4L2 on Linux, AVF on macOS, MSMF on Windows) or OpenCV instead.

Known Limitations:

• Setting camera controls (brightness, contrast, etc.) is not supported.
• Setting the FrameFormat is not supported.
• Dropping the device instance may cause a panic in certain scenarios.

use nokhwa::backends::capture::GStreamerCaptureDevice;
use nokhwa::utils::{CameraFormat, Resolution};

// Example: Creating a device with specific resolution
let cam_fmt = CameraFormat::new(Resolution::new(1280, 720));
let mut device = GStreamerCaptureDevice::new(0, Some(cam_fmt)).unwrap();

// Example: Creating a device with default settings (640x480 @ 15 FPS, MJPEG)
let mut device = GStreamerCaptureDevice::new(0, None).unwrap();

The MediaFoundationCaptureDevice is the backend implementation for Windows (Windows 7 or newer) using the Media Foundation API. It implements the CaptureTrait.

Quirks & Limitations

• Platform Specificity: While it may build on non-Windows platforms, the backend will be empty and return errors for any operation if not running on Windows.
• Device Naming: Device names may contain invalid characters due to conversion from UTF16.
• Device Identification: The symbolic link for the device is stored in the misc attribute of the CameraInformation.
• Lifecycle: initialize and de_initialize are automatically called when the struct is created (new) or dropped.

use nokhwa::MediaFoundationCaptureDevice;
use nokhwa::types::{CameraIndex, RequestedFormatType};
use nokhwa::core::camera_format::CameraFormat;

// Example initialization (conceptual)
// let device = MediaFoundationCaptureDevice::new(&index, requested_format)?;