image-quality

If you are developing the library and need to add a new TensorFlow dataset or modify a zip file location, you must update the URL checksums following these steps:

1. Place your dataset configuration file (e.g., live_iqa.py) into the tensorflow_datasets folder (typically located in ${HOME}/.local/lib/python3.8/site-packages if installed with the --user flag).
2. Update the tensorflow_datasets/__init__.py file to import your new dataset (e.g., from .image.live_iqa import LiveIQA).
3. Create a new checksum file and run the download/prepare script:

touch url_checksums/live_iqa.txt
python -m tensorflow_datasets.scripts.download_and_prepare \
    --register_checksums \
    --datasets=live_iqa

4. Copy the generated checksum from the resulting live_iqa.txt file into your project's url_checksums folder.

The image-quality library is available on PyPI. You can install it using pip.

Requirements:

• Python 3.8

pip install image-quality

DIQA requires a specific preprocessing step where the image is converted to grayscale and a low-pass filter is applied to isolate high-frequency components (distortions). The filter is created by blurring the image, downscaling it by 1/4, and upscaling it back to the original size, then subtracting this low-frequency component from the grayscale image.

def image_preprocess(image: tf.Tensor) -> tf.Tensor:
    image = tf.cast(image, tf.float32)
    image = tf.image.rgb_to_grayscale(image)
    # Apply low-pass filter logic
    image_low = gaussian_filter(image, 16, 7 / 6)
    image_low = rescale(image_low, 1 / 4, method=tf.image.ResizeMethod.BICUBIC)
    image_low = tf.image.resize(image_low, size=image_shape(image), method=tf.image.ResizeMethod.BICUBIC)
    return image - tf.cast(image_low, image.dtype)

The Objective Error Model is trained using a custom loss function that calculates the Mean Squared Error of the product between the reliability map and the error map. Because of this custom logic, a custom training loop using tf.GradientTape is required.

# Custom loss: MSE of (error * reliability)
def loss(model, x, y_true, r):
    y_pred = model(x)
    return tf.reduce_mean(tf.square((y_true - y_pred) * r))

# Custom gradient calculation
def gradient(model, x, y_true, r):
    with tf.GradientTape() as tape:
        loss_value = loss(model, x, y_true, r)
    return loss_value, tape.gradient(loss_value, model.trainable_variables)

# Training loop snippet
for I_d, e_gt, r in train_dataset:
    loss_value, gradients = gradient(objective_error_model, I_d, e_gt, r)
    optimizer.apply_gradients(zip(gradients, objective_error_model.trainable_weights))

The Subjective Score Model is a regressor trained on the features extracted by the convolutional layers of the Objective Error Model. It can be trained using the standard Keras .fit() method.

# Define the regressor using the output of the CNN feature extractor (f)
v = GlobalAveragePooling2D(data_format='channels_last')(f)
h = Dense(128, activation='relu')(v)
h = Dense(1)(h)
subjective_error = tf.keras.Model(input, h, name='subjective_error')

# Prepare dataset for .fit() (returns input and target)
def calculate_subjective_score(features):
    I_d = image_preprocess(features['distorted_image'])
    mos = features['dmos']
    return (I_d, mos)

train_ds = ds.map(calculate_subjective_score)
subjective_error.fit(train_ds, epochs=1)

The imquality.datasets module provides TensorFlow dataset builders following the tensorflow-datasets interface. You can download and prepare datasets like LiveIQA using the following pattern:

import imquality.datasets

# Initialize the builder
builder = imquality.datasets.LiveIQA()

# Download and prepare the data (this may take several minutes)
builder.download_and_prepare()

# Convert to a tf.data.Dataset
ds = builder.as_dataset(shuffle_files=True)['train']

# Note: Because images have different shapes, use a batch size of 1
ds = ds.shuffle(1024).batch(1)

To use the dataset builders provided in the imquality package, install the library with the [dataset] extra.

!pip install --user "image-quality[dataset]>=1.2.4"

After installation, you can use the imquality.brisque module to calculate a quality score for a given image. This requires the PIL (Pillow) library to load the image.

1. Import imquality.brisque and PIL.Image.
2. Open your image using PIL.Image.open().
3. Pass the image object to brisque.score() to receive a numerical quality score.

import imquality.brisque as brisque
import PIL.Image

path = 'path/to/image'
img = PIL.Image.open(path)
score = brisque.score(img)
print(score)

The image-quality service is built from the local context using the Dockerfile located at docker/test/Dockerfile.

When building, the following parameters are used:

• Target: The build targets the base stage.
• Build Argument: PYTHON_DOCKER_VERSION is passed as a build argument to control the Python version.
• Image Tag: The resulting image is tagged using the ${VERSION} environment variable.

services:
  image-quality:
    image: ocampor/image-quality:${VERSION}
    build:
      context: .
      dockerfile: docker/test/Dockerfile
      args:
        PYTHON_DOCKER_VERSION: ${PYTHON_DOCKER_VERSION}
      target: base

The notebook service provides a Jupyter environment. It uses the ocampor/notebook image and exposes port 8000.

To secure the notebook, you must provide a JUPYTER_PASS environment variable. The service also maps local directories to the container:

• ./notebooks is mapped to /home/ocampor/notebooks inside the container.
• A local data directory (e.g., /Users/ricardoocampo/Data) is mapped to /data inside the container.

services:
  notebook:
    image: ocampor/notebook
    environment:
      - JUPYTER_PASS=${JUPTER_PASS}
    ports:
      - 8000:8000
    volumes:
      - ./notebooks:/home/ocampor/notebooks
      - /Users/ricardoocampo/Data:/data

Once the subjective_error model is trained, use the .predict() method on a pre-processed image to obtain the predicted subjective score.

sample = next(iter(ds))
I_d = image_preprocess(sample['distorted_image'])
target = sample['dmos'][0]

prediction = subjective_error.predict(I_d)[0][0]
print(f'the predicted value is: {prediction:.4f} and target is: {target:.4f}')

To prevent the model from failing on homogeneous (blurry) regions, a reliability map $\mathbf{r}$ is used. It assigns higher values to textured areas and lower values to blurry ones using a sigmoid-based function.

def reliability_map(distorted: tf.Tensor, alpha: float) -> tf.Tensor:
    assert distorted.dtype == tf.float32, 'The Tensor must by of dtype tf.float32'
    return 2 / (1 + tf.exp(- alpha * tf.abs(distorted))) - 1

# Use the average reliability map for score prediction
def average_reliability_map(distorted: tf.Tensor, alpha: float) -> tf.Tensor:
    r = reliability_map(distorted, alpha)
    return r / tf.reduce_mean(r)