The Flink Kubernetes Operator's autoscaler determines the TRUE_PROCESSING_RATE (TPR) to decide how to scale a job. TPR represents how fast a vertex could process at full utilization. The autoscaler calculates this using two distinct methods—Busy-Time Estimate and Backpressure Estimate—and selects the most appropriate one based on specific rules.
1. Busy-Time Estimate
This method calculates capacity based on how much time a vertex spends 'busy'. If a vertex achieves rate R while busy a fraction b of the time, the estimated capacity is R / b.
The calculation depends on the job.autoscaler.metrics.busy-time.aggregator configuration:
busy-time.aggregator | busyTimeAvg (denominator) | inputRateForTpr (numerator) |
|---|
MAX or MIN (default MAX) | windowed mean of LOAD × 1000 | windowed mean of NUM_RECORDS_IN_PER_SECOND (fallback to NUM_RECORDS_IN) |
AVG | windowed rate of ACCUMULATED_BUSY_TIME / parallelism | windowed rate of NUM_RECORDS_IN |
2. Backpressure Estimate
Used primarily for sources when busy-time becomes unreliable due to sustained backpressure. It divides the achieved rate by the fraction of time the vertex was not blocked:
OBSERVED_TPR = numRecordsInPerSecond / (1 − backPressuredTimeMsPerSecond / 1000)
This estimate is only considered valid for sources that are 'catching up' (where LAG is greater than or equal to the rate multiplied by the job.autoscaler.observed-true-processing-rate.lag-threshold, which defaults to 30s).
| Observation State | OBSERVED_TPR Value |
|---|
Catching up (backpressure < 1000 ms/s) | The formula above |
Idle (numRecordsInPerSecond = 0) | +∞ (allows scale-down) |
Fully backpressured (≥ 1000 ms/s) | NaN |
| Otherwise | Historical average (or NaN if below min-observations, default 2) |
3. Estimate Selection Logic
The autoscaler selects the final TPR using the following priority rules:
- If
observedTprAvg is NaN $\rightarrow$ use busyTimeTpr. - If
busyTimeTpr is NaN or +∞ $\rightarrow$ use OBSERVED_TPR. - If
busyTimeTpr > observedTprAvg × (1 + switch-threshold) (default 0.15) $\rightarrow$ use OBSERVED_TPR. - Otherwise $\rightarrow$ use
busyTimeTpr.
Summary: Most vertices use busyTimeTpr. The autoscaler switches to the more conservative OBSERVED_TPR for heavily backpressured sources where idle time relative to busy time exceeds the switch-threshold.
// Busy-Time Formula
busyTimeTpr = inputRateForTpr / (busyTimeAvg / 1000)
// Backpressure Formula
OBSERVED_TPR = numRecordsInPerSecond / (1 − backPressuredTimeMsPerSecond / 1000)
