Digital Video Introduction

A generic video codec operates through three main stages to compress video data:

1. Image Partitioning: Dividing the frame into partitions, sub-partitions, slices (or tiles), and macroblocks (or coding tree units). This allows for more precise predictions (e.g., small partitions for moving parts and large partitions for static backgrounds).
   • HEVC maximum partition size: 64x64
   • AVC maximum partition size: 16x16
   • Sub-partitions can reach as small as 4x4.
2. Prediction: Making predictions on the partitions.
   • Inter prediction (temporal redundancy): Sends motion vectors and the residual.
   • Intra prediction (spatial redundancy): Sends the prediction direction and the residual.
3. Transformation: Transforming the residual block (calculated as predicted partition - actual partition) to identify which pixels can be discarded while maintaining overall quality.

Entropy coding is the final step in video compression, applied after quantization to compress data losslessly.

Variable Length Coding (VLC) works by assigning shorter binary codes to symbols with higher probabilities and longer codes to symbols with lower probabilities.

Key Requirements:

• Prefix Property: Every binary code must have a unique prefix so that the decoder can identify symbol boundaries without separators.
• Symbol-Code Table: Both the encoder and the decoder must possess the same symbol-to-code mapping table to ensure successful reconstruction.

Example:

For the word eat with specific symbol probabilities, instead of using fixed 8-bit characters (24 bits total), VLC can represent the sequence using much fewer bits (e.g., 7 bits) by using variable-length binary strings.

|                | a   | e   | r   | t    |
|----------------|-----|-----|-----|------|
| probabilità    | 0,3 | 0,3 | 0,2 | 0,2  |
| codice binario | 0   | 10  | 110 | 1110 |

DRM is used to protect digital media from unauthorized distribution. While token-based authentication protects access to a stream, DRM protects the content itself via encryption.

Major DRM Systems

• FPS (FairPlay Streaming) - Apple
• PR (PlayReady) - Microsoft
• WV (Widevine) - Google

The DRM Workflow

1. Encryption: Content ($C_1$) is encrypted by a DRM system using a symmetric key ($K_1$) to produce encrypted content ($C'_1$).
2. Key Exchange: The player ($P_1$) on a device ($D_1$) sends its public key ($PUK_1$) to the DRM system. The DRM system returns the symmetric key $K_1$ encrypted with the player's public key: $K_{1P1D1} = ext{enc}(K_1, PUK_1)$.
3. Decryption: The device uses its protected private key ($PRK_1$) to decrypt the symmetric key: $K_1 = ext{dec}(K_{1P1D1}, PRK_1)$.
4. Playback: The player uses $K_1$ to decrypt the content: $P_1. ext{play}( ext{dec}(C'_1, K_1))$.

Protection can be implemented via software or more securely via hardware (e.g., storing keys in a secure chip/SoC where they cannot be exported from memory).

Chroma subsampling is a technique used to reduce video data size by encoding chrominance (color) at a lower resolution than luminance (brightness). This exploits the human eye's lower sensitivity to color detail.

Subsampling schemes are expressed as a three-part ratio a:x:y relative to a block of a x 2 luminance pixels:

• a: Horizontal sampling reference (usually 4).
• x: Number of chrominance samples in the first row of a pixels.
• y: Number of changes in chrominance samples between the first and second rows of a pixels.

Common schemes used in modern codecs:

• 4:4:4: No subsampling (highest quality).
• 4:2:2, 4:1:1, 4:2:0, 4:1:0, 3:1:1.

Note on 4:1:0: This is an exception that provides a single chrominance sample within every 4 x 4 luma block.

Impact on Data Size: Using YCbCr 4:2:0 can reduce the data size by half compared to uncompressed RGB. For example, a 720p 30fps video drops from 24 bits per pixel to 12 bits per pixel, reducing a 1-hour file from 278 GB to 139 GB.

Temporal redundancy refers to repetitions in time between consecutive frames. This is addressed using inter prediction techniques to reduce the number of bits required to encode a sequence of frames.

Key methods include:

• Delta Encoding: Subtracting one frame from another (e.g., frame 0 - frame 1) to encode only the residual (the difference).
• Motion Estimation: Dividing a frame into partitions (blocks) and finding where those blocks moved in the next frame. The movement is described by motion vectors (x, y values).
• Motion Vector Difference: To save even more bits, codecs can encode only the difference between the last known block position and the predicted position.

Note that motion estimation is rarely perfect due to 3D movement (e.g., objects changing size as they move to the background), so a residual is still calculated to correct the prediction.

# Conceptual example of motion vector difference
# If a block moves from x=0, y=25 to x=7, y=26:
# Motion Vector = (7, 1)
# Encoded as the difference: (7-0, 26-25) = (7, 1)

Chroma subsampling is a technique used to reduce the amount of data required for an image by using a lower resolution for color components than for the brightness (luma) component. This is possible because the human eye is less sensitive to color detail.

Subsampling schemes are typically expressed as a ratio a:x:y, which defines the color resolution relative to a block of a * 2 luma pixels:

• a: Horizontal width (usually 4).
• x: Number of color samples in the first row.
• y: Number of color changes between the first and second row.

Note: In the 4:1:0 scheme, a single color is chosen for every 4 x 4 luma resolution block.

Commonly used formats in modern codecs include:

• 4:4:4 (No subsampling)
• **4:2:2$
• **4:1:1$
• **4:2:0$
• **4:1:0$
• **3:1:1$

Using YCbCr 4:2:0 can significantly reduce file size. For example, a 720p/30fps video that might require 278GB of raw data can be reduced to approximately 139GB by using 12 bits per pixel instead of 24.

Arithmetic Coding is a sophisticated lossless compression method that represents an entire sequence of symbols as a single fractional number within a range (typically between 0 and 1).

How it works:

1. Interval Partitioning: The entire range [0, 1) is partitioned into sub-intervals based on the probability of each symbol.
2. Encoding: For each symbol in the stream, the current interval is narrowed down to the sub-interval corresponding to that symbol. This process repeats for every symbol.
3. Result: The final compressed data is a single number (e.g., 0.36) that falls within the final narrowed interval.
4. Decoding: The decoder uses the number and the original probability table to iteratively determine which sub-intervals the number falls into, thereby reconstructing the original symbol stream.

Requirement: Like VLC, both the encoder and decoder must have access to the symbol probability table.

Note: Many modern video codecs use advanced versions of this, such as CABAC (Context-adaptive binary arithmetic coding).