WALT (WALT Latency Timer)

WALT measures screen latency by detecting physical color changes (specifically between black and white) using a photodiode.

On Android, the WALT app uses View.setBackgroundColor to trigger a color change. The latency reported is the time elapsed from the start of the next frame rendering event (governed by the Android Choreographer.FrameCallback) until the screen physically transitions to the new color.

Key considerations for measurements:

• Directionality: The physical transition time may differ depending on whether the screen is transitioning from black to white or white to black.
• Hardware Variance: Transition speeds and the timing of different screen sections vary significantly based on screen technology and device models.
• Setup for Accuracy: For consistent results, measurements should be taken with the device set to maximum brightness and the WALT hardware positioned in the middle of the screen.

The WALT app calculates latency by analyzing the relationship between motion events (coordinates and timestamps) and laser beam interruption events.

1D Calculation (Standard)

For standard measurements, the app assumes the laser beam is parallel to the x-axis and focuses on the vertical coordinate ($y$) as a function of time $y(t)$.

• The app identifies points where the finger enters or exits the laser beam (marked as 'x' in plots).
• It searches for a time shift $S$ that minimizes the standard deviation of $y(t + S)$.
• A minimal standard deviation indicates that the 'x' marks align as closely as possible to a horizontal straight line, effectively compensating for the clock divergence between touch and laser events.
• The final reported latency is the average of the two time shifts calculated for the two sides of the finger (the upper and lower lines of 'x' marks).

2D Calculation (ChromeOS)

For ChromeOS, a 2D extension of this method is used via a Python script. This method requires a considerable spread of the collected $x$ coordinates to function correctly.

WALT implements USB communication using the Linux API declared in linux/usbdevice_fs.h. It currently uses a BULK interface (typically exposed when Teensy code is compiled in "Serial" mode).

There are two ways to perform communication via usbdevice_fs:

1. Asynchronous way: Uses USBDEVFS_SUBMITURB followed by USBDEVFS_REAPURB (which blocks until a URB is available).
2. Synchronous way: Uses USBDEVFS_BULK with a struct usbdevfs_bulktransfer configuration.

Implementation Details:

• All timestamps in the C code are recorded in microseconds.
• Timestamped messages are single-byte.
• The C code is compatible with both Android JNI and Linux.

// Async way
ioctl(fd, USBDEVFS_SUBMITURB, urb);
// followed by
ioctl(fd, USBDEVFS_REAPURB, &urb); // Blocks until there is a URB to read.

// Sync way
struct usbdevfs_bulktransfer  ctrl;
ctrl.ep = endpoint;
ctrl.len = length;
ctrl.data = buffer;
ctrl.timeout = timeout; // [milliseconds] Will timeout if there is nothing to read
int ret = ioctl(fd, USBDEVFS_BULK, &ctrl);

WALT uses a three-step algorithm to synchronize a local clock with a remote clock, accounting for asymmetric USB latency. The goal is to bound the unknown clock difference E within a range minE < E < maxE.

Step 1: Rough Sync

1. Record local time T0.
2. Instruct the remote device to zero its clock.
3. Wait for confirmation from the remote.
4. Set maxE = current_time() - T0.
5. All subsequent local time is measured relative to T0.

At this stage, the remote clock lags behind the local clock by E, where 0 = minE < E < maxE.

Step 2: Find better lower bound (minE)

To increase minE, send messages from local to remote. The remote replies with its timestamp (t_remote) of when it received the message. The local machine records the time right before sending (t_local).

• Logic: E > t_local - t_remote (since travel time > 0).
• Action: set minE to max(minE, t_local - t_remote).
• Note: Send a batch of messages with random small delays before retrieving timestamps to mitigate kernel/hardware buffering delays.

Step 3: Find better upper bound (maxE)

To decrease maxE, the remote device sends messages to the local machine. The remote includes its timestamp (t_remote) of when the message was sent, and the local machine records the time of receipt (t_local).

• Logic: E < t_local - t_remote.
• Action: set maxE = min(maxE, t_local - t_remote).

Comparison with NTP

Unlike NTP, which assumes symmetric travel time (latency), WALT is designed for asymmetric USB communication. While the resulting synchronization interval deltaE = maxE - minE may be larger than NTP's on symmetric networks, WALT provides guaranteed bounds even when asymmetry is present.

WALT measures audio latency as the time elapsed between an application's decision to output a tone and the moment that tone is detected via the headphone jack (for playback), or similarly for microphone latency (recording).

Playback Latency

WALT reports the time from the Enqueue() call in the player callback until a detectable signal is present on the wire.

Note: This measurement deliberately omits the time between the application's decision to output a tone and the actual Enqueue() call. This omission is intended to reduce variance, as that interval is often uniformly distributed between 0 and the buffer length (e.g., 5 ms on a Nexus 5).

To ensure accurate measurements and avoid 'warm-up' latency, it is recommended to constantly enqueue buffers containing silence. The WALT app follows this pattern.

Recording Latency

WALT reports the time from the moment the last frame in a buffer was recorded until the recorder callback receiving that buffer is executed.

Electronics Implementation Notes

• Teensy 3.1 Compatibility: If using a Teensy 3.1 instead of an LC, the screen photodiodes cannot be read directly. You must use a buffer opamp. While a single photodiode with a 1.5M resistor can work, it may hit the input impedance limits of the Teensy.
• Photodiode Orientation: The BPW34 photodiode has a small protrusion on the cathode pin (cathode marker). The anode has a white dot near it.
• Laser Setup: The laser photodiode setup uses an internal pullup resistor (~20k), enabled via pinMode(PD_LASER_PIN, INPUT_PULLUP);.

Mechanical Requirements

The enclosure must be made from a non-conductive and non-transparent material.

Enclosure Options:

• Production Version (r0.7): Uses WALT_recessed_enclosure.stl or .step files. Requires four M3x6 screws (McMaster-Carr PN 92000A116) and M3 hex nuts (McMaster-Carr PN 91828A211).
• Desktop 3D Printing: A simpler version is available via OnShape or the WALT_3DP_enclosure_v7.stl file.
• 2D Cutting: A minimal enclosure for 5-8mm thick substrates is available via OnShape.