PhyCV Documentation

PhyCV is a physics-inspired computer vision library that emulates the propagation of light through physical media followed by coherent detection. It currently provides three main algorithm suites, each available in CPU and GPU versions:

1. Phase-Stretch Transform (PST): A computationally efficient edge and texture detection algorithm designed for high performance in visually impaired images.
2. Phase-Stretch Adaptive Gradient-field Extractor (PAGE): An algorithm for detecting edges and their orientations at various scales using diffraction equations. It uses a bank of filters to detect directional edges, allowing it to pick up structural details that PST might miss.
3. Vision Enhancement via Virtual diffraction and coherent Detection (VEViD): A low-light and color enhancement algorithm that treats a digital image as a spatially varying light field and applies processes akin to diffraction and coherent detection.

The PAGE class is used for edge extraction using physics-inspired gradient fields. You can use the high-level run method to execute the entire pipeline (loading, kernel initialization, application, and edge creation) in one call, or manually control each step.

CPU Implementation

To use the CPU version, instantiate PAGE and provide direction_bins. If h (height) and w (width) are not provided during initialization, they will be inferred from the image loaded via load_img.

GPU Acceleration

The PAGE_GPU class (available in phycv/page_gpu.py) provides significant acceleration. It follows a similar API to the CPU version but with these key differences:

• Device Selection: You must specify a device (e.g., 'cuda') during instantiation, similar to PyTorch.
• Data Types: Image I/O uses torchvision and matrix operations use torch instead of numpy or opencv.
• Parallelism: init_kernel and apply_kernel use broadcasting to process direction bins in parallel.
• Output: Results are returned as torch.Tensor objects located on the specified GPU device.

from phycv.page import PAGE

# Initialize with direction bins
page = PAGE(direction_bins=8)

# Run the full pipeline on a single image
# Parameters: img_file, mu_1, mu_2, sigma_1, sigma_2, S1, S2, sigma_LPF, thresh_min, thresh_max, morph_flag
result = page.run(
    img_file='path/to/image.jpg',
    mu_1=0.1, mu_2=0.2, sigma_1=1.0, sigma_2=1.0, S1=1.0, S2=1.0,
    sigma_LPF=2.0, thresh_min=0.5, thresh_max=0.8, morph_flag=1
)

The VEVID class implements Vision Enhancement via Virtual diffraction and coherent Detection. It provides methods for image loading, kernel initialization, and applying the enhancement kernel to the V (Value) channel of an HSV image.

Workflow

1. Instantiate: Create a VEVID instance. You can optionally specify h (height) and w (width) during initialization. If left as None, they are inferred from the loaded image.
2. Load Image: Use load_img to provide either an img_file path or an img_array. The class automatically converts the image from RGB to HSV.
3. Initialize Kernel: Call init_kernel(S, T) where S is the phase scale and T is the phase variance.
4. Apply Enhancement: Use apply_kernel with regularization constant b and phase activation gain G.
   • Set color=True for color enhancement.
   • Set color=False (default) for low-light enhancement.
   • Set lite=True to use an approximated version of the algorithm.

Alternatively, use the convenience methods run (full algorithm) or run_lite (accelerated version) to wrap these steps.

# Full algorithm workflow
vevid = VEVID()
result = vevid.run(img_file='path/to/image.jpg', S=S_param, T=T_param, b=b_param, G=G_param, color=False)

# Accelerated (lite) workflow
vevid_lite = VEVID()
result = vevid_lite.run_lite(img_file='path/to/image.jpg', b=b_param, G=G_param, color=False)

The PAGE class provides the following methods for edge extraction:

• __init__(self, direction_bins, h=None, w=None): Initializes the extractor. h and w are optional and can be set to None to be determined by the image shape.
• load_img(self, img_file=None, img_array=None): Loads an image from a file or a numpy array. Converts RGB to grayscale. If h and w were not set in __init__, they are set here.
• init_kernel(self, mu_1, mu_2, sigma_1, sigma_2, S1, S2): Initializes the PAGE kernels based on physical parameters. In the CPU version, this is done serially per frequency bin.
• apply_kernel(self, sigma_LPF, thresh_min, thresh_max, morph_flag): Denoises the image with a low-pass filter (sigma_LPF), applies the kernels, and optionally applies morphological operations if morph_flag == 1 using thresh_min and thresh_max.
• create_page_edge(self): Generates a weighted color image to visualize the directionality of the extracted edges.
• run(self, img_file, mu_1, mu_2, sigma_1, sigma_2, S1, S2, sigma_LPF, thresh_min, thresh_max, morph_flag): A convenience wrapper that executes load_img, init_kernel, apply_kernel, and create_page_edge in sequence.

The PST class implements the Phase-Stretch Transform. You can use the high-level run method to execute the entire pipeline in one call, or manually control the lifecycle using load_img, init_kernel, and apply_kernel.

Workflow

1. Initialization: Instantiate PST(h=None, w=None). If h and w are provided, the input image will be reshaped to these dimensions. If None, dimensions are inferred from the loaded image.
2. Loading: Use load_img(img_file=..., img_array=...). RGB images are automatically converted to grayscale.
3. Kernel Setup: Call init_kernel(S, W) where S is phase strength and W is warp strength.
4. Execution: Call apply_kernel(...) to denoise (via sigma_LPF), apply the kernel, and optionally perform morphological operations if morph_flag == 1 using thresh_min and thresh_max.

from phycv import PST

# High-level approach
pst = PST()
pst.run(
    img_file='path/to/image.png',
    phase_strength=S,
    warp_strength=W,
    sigma_LPF=sigma,
    thresh_min=t_min,
    thresh_max=t_max,
    morph_flag=1
)