imageZMQ

The REQ/REP (Request/Reply) pattern is the default in imageZMQ. In this mode, the ImageSender (client) sends an image and waits for an 'OK' reply from the ImageHub (server) before sending the next frame.

Key Characteristics:

• Blocking: sender.send_image() is a blocking operation. If the Hub is disconnected or unresponsive, the sender will freeze until a reply is received.
• Reliability: Provides explicit acknowledgement that every frame was received.
• Topology: Multiple senders can connect to a single Hub. Senders only need to know the Hub's address.
• Identification: Each message is a (text_message, image) tuple. Using a unique identifier (like a hostname) in the text portion allows the Hub to display multiple streams in separate OpenCV windows.

# Sender (RPi)
import imagezmq
sender = imagezmq.ImageSender(connect_to='tcp://<HUB_IP>:5555')
sender.send_image('rpi_name', image)

# Receiver (Hub)
import imagezmq
import cv2
image_hub = imagezmq.ImageHub()
rpi_name, image = image_hub.recv_image()
cv2.imshow(rpi_name, image)
image_hub.send_reply(b'OK')

In a standard PUB/SUB pattern, a receiver might fall behind if its processing (e.g., CNN object detection or motion detection) is slower than the incoming frame rate. This causes the receiver to process stale frames sitting in the ZMQ socket queue rather than the most recent ones.

To ensure a subscriber always processes the most recent frame without the overhead of constant reconnecting, use a multithreaded approach:

1. Use a dedicated IO thread to continuously receive frames from the socket.
2. Use a processing thread (the main thread) to consume only the latest frame.

The VideoStreamSubscriber helper class implements this pattern by using a threading.Event to signal when new data is available, allowing the main thread to grab the latest frame and skip older ones in the queue.

# Conceptual pattern for fast subscriber:
# 1. Dedicated IO thread calls receiver.recv_jpg() continuously
# 2. Main thread calls receive() to get the latest data

The PUB/SUB (Publish/Subscribe) pattern is used when you want non-blocking image streaming.

Key Characteristics:

• Non-blocking: The sender does not wait for a reply. If no subscribers are connected, images are discarded immediately and execution continues.
• Topology: The direction of connection is reversed. The ImageHub (subscriber) must know the address of every ImageSender (publisher).
• Configuration: You must set REQ_REP=False in both the ImageSender and ImageHub constructors.
• Setup: The ImageHub is initialized with the first sender's address via open_port, and subsequent senders are added using the .connect() method.

# Sender (RPi)
import imagezmq
sender = imagezmq.ImageSender(connect_to='tcp://*:5555', REQ_REP=False)
sender.send_image('rpi_name', image)

# Receiver (Hub)
import imagezmq
import cv2
image_hub = imagezmq.ImageHub(open_port='tcp://192.168.1.100:5555', REQ_REP=False)
image_hub.connect('tcp://192.168.0.101:5555')
rpi_name, image = image_hub.recv_image()
cv2.imshow(rpi_name, image)

imageZMQ is designed to distribute computer vision workloads across multiple devices in a network. A common pattern is to use low-power edge devices (like Raspberry Pis) to capture images and perform lightweight processing, while using a more powerful central hub (like a Mac or Linux PC) to perform heavy computation and long-term storage.

Recommended Architecture Pattern

1. Edge Nodes (e.g., Raspberry Pi):

   • Capture images via camera (e.g., PiCamera).
   • Perform lightweight preprocessing (rotation, cropping, grayscale conversion, thresholding).
   • Implement local logic to detect events (e.g., motion detection or status changes).
   • Optimization: Only transmit images via imageZMQ when a significant event occurs to minimize network load and avoid SD card wear on the edge device.

2. Central Hub (e.g., Mac/Linux):

   • Receive image streams and status messages via imageZMQ.
   • Perform complex processing (e.g., feature extraction, digit classification, ROI analysis).
   • Store images and metadata in a database.
   • Act as a single processing hub for multiple edge nodes (a single hub can handle 8 or more concurrent image streams).

imageZMQ supports two messaging patterns for transporting images between ImageSender and ImageHub. The choice depends on whether you require guaranteed delivery and blocking behavior (REQ/REP) or high-throughput, non-blocking behavior (PUB/SUB).

REQ/REP (Request/Reply)

Default Mode. This pattern guarantees that each image is delivered. The sender waits for a confirmation (a REP) from the recipient before sending the next image, making the sender "blocking."

• Best for: Many-to-one scenarios (e.g., many Raspberry Pis sending to one central Hub). The Hub does not need to know the IP addresses of the senders.
• Pros: Verified delivery; Hub doesn't need sender addresses; supports multiple senders simultaneously.
• Cons: Sender blocks if the Hub is unavailable; if the Hub restarts, all Senders must also restart.

PUB/SUB (Publish/Subscribe)

This is a non-blocking pattern where the sender does not expect confirmation. The sender will continue to stream images even if no recipients are listening.

• Best for: Many-to-many scenarios. It allows for high-speed streaming where the sender's execution is not tied to the receiver's speed.
• Pros: Non-blocking; Senders do not need to restart if the Hub restarts; supports many-to-many relations.
• Cons: Delivery is not guaranteed; the Hub must know the address of every Sender in advance and explicitly subscribe to them; slow subscribers can cause ZMQ queue build-up (the "Suicidal Snail" problem).

# Example of how to switch modes during instantiation
# REQ/REP is the default (REQ_REP=True)

# For REQ/REP mode:
sender = ImageSender(REQ_REP=True)
hub = ImageHub(REQ_REP=True)

# For PUB/SUB mode:
sender = ImageSender(REQ_REP=False)
hub = ImageHub(REQ_REP=False)

To build a system where multiple cameras send images to a headless server for processing (like motion detection) without blocking the main pipeline, you can combine imagezmq.ImageHub (REQ/REP) with imagezmq.ImageSender (PUB/SUB).

1. The Cameras (RPi): Use imagezmq.ImageSender in default REQ/REP mode to send images to the hub.
2. The Hub (Server):
   • Use imagezmq.ImageHub() to receive images from cameras via REQ/REP.
   • Use imagezmq.ImageSender(connect_to='tcp://*:PORT', REQ_REP=False) to create a PUB server. This allows you to broadcast the processed images to any number of remote monitors without affecting the camera-to-hub communication.
3. The Monitor (Client): Connect to the PUB server to view the stream.

import cv2
import imagezmq

def processImage(image):
    # Perform processing (e.g., motion detection)
    pass

# Create a hub for receiving images from cameras (REQ/REP)
image_hub = imagezmq.ImageHub()

# Create a PUB server to broadcast images for monitoring (PUB/SUB)
stream_monitor = imagezmq.ImageSender(connect_to = 'tcp://*:5566', REQ_REP = False)

while True:
    rpi_name, image = image_hub.recv_image()
    image_hub.send_reply(b'OK') # Acknowledge receipt
    processImage(image)
    stream_monitor.send_image(rpi_name, image) # Broadcast to monitors