Napkin Math

To run the active Criterion benchmark suite with the correct optimization levels and Linux tuning, use the ./run command. Note that the wrapper uses sudo internally. If you are on a locked-down cloud image, you may need to run sudo sysctl -w kernel.perf_event_paranoid=-1 once before invoking the benchmark.

Warning: Do not run benchmarks in debug mode, as you will not get accurate performance numbers.

./run --bench napkin_math

When performing back-of-the-envelope calculations, follow these principles:

• Don't overcomplicate: Avoid basing calculations on more than 6 assumptions.
• Keep the units: Use units as checksums to ensure accuracy (e.g., converting KiB to TiB).
• Calculate with exponents: Focus on the exponent ($e$ in $c \times 10^e$) to get within an order of magnitude. Single-digit coefficients matter less than the exponent.
• Perform Fermi decomposition: Break down complex questions into smaller, guessable components (e.g., to estimate log storage cost, estimate log line size $\rightarrow$ logs per second $\rightarrow$ total volume $\rightarrow$ cost).

The blob_storage Criterion group is opt-in and requires credentials. Both GCS and S3 paths use the AWS S3 SDK.

GCS Configuration:

• Set NAPKIN_GCS_BUCKET to specify the bucket.
• Set NAPKIN_GCS_ACCESS_KEY and NAPKIN_GCS_SECRET_KEY for authentication.

S3 Configuration:

• Set NAPKIN_S3_BUCKET to specify the bucket.
• Use NAPKIN_S3_PROFILE to specify the local AWS profile (defaults to tpuf-test).
• Use NAPKIN_S3_REGION to specify the region (defaults to us-west-2).

To use SQL magic commands within a Jupyter notebook to interact with a MySQL database, you need to install the required drivers and extensions, then load the sql extension. This setup allows you to execute SQL queries directly in cells using the %sql magic command.

!pip3 install mysqlclient sqlalchemy jupyter_contrib_nbextensions
!jupyter contrib nbextension install --user
!jupyter nbextension enable python-markdown/main

%load_ext sql
%sql mysql://root@localhost/napkin

To sweep get, put, if_none_match, and put_if_match across a size ladder (defaulting to 8 KiB .. 8 MiB), use the provided scripts.

Commands:

• For S3: ./script/blob-latency s3
• For GCS: ./script/blob-latency gcs

Tunable Environment Variables:

• NAPKIN_BLOB_LATENCY_OPS: Specify which operations to run.
• NAPKIN_BLOB_LATENCY_SIZES: Specify the size ladder.
• NAPKIN_BLOB_LATENCY_IF_NONE_MATCH_SIZES: Specify sizes for the if_none_match operation.
• NAPKIN_BLOB_LATENCY_ROW_SECONDS: Set the wall-clock budget per row (default is 300 seconds).
• NAPKIN_BLOB_LATENCY_SAMPLE_CAP (or NAPKIN_BLOB_LATENCY_SAMPLES): Cap the number of samples collected.

Use these scripts to measure latency for specific alignment shapes over a 128 x 1 GiB object pool:

• 128 KiB alignment: ./script/blob-random-range-latency
• 8 MiB alignment: ./script/blob-random-range-latency-8m
• 8 MiB alignment (multipart): ./script/blob-random-range-latency-8m-multipart (this seeds source objects via multipart upload with 8 MiB parts).

Use these ballpark compression ratios for estimating storage and bandwidth needs. Note that compression speed typically decreases as the ratio increases (e.g., a 2x increase in ratio might decrease performance by 10x).

| What        | Compression Ratio |
| ----------- | ----------------- |
| HTML        | 2-3x              |
| English     | 2-4x              |
| Source Code | 2-4x              |
| Executables | 2-3x              |
| RPC         | 5-10x             |
| SSL         | -2%               |

Use these functions to benchmark different map-based storage strategies:

• pointerMap(nCells int): Uses map[Rank]*LargeCell.
• pointerMapSmallCells(nCells int): Uses map[Rank]*SmallCell.
• pointerMapSmallerCells(nCells int): Uses map[Rank]*SmallerCell.
• valueMap(nCells int): Uses map[Rank]LargeCell (stores values directly in the map).
• pointerMapIntegerIndex(nCells int): Uses map[int]*LargeCell.
• pointerMapIntegerIndexSmallCells(nCells int): Uses map[int]*SmallCell.
• pointerMapIntegerIndexSmallerCells(nCells int): Uses map[int]*SmallerCell.
• pointerMapIntegerIndexSmallCellsValue(nCells int): Uses map[int]SmallCell (stores values directly).

Use the following functions to benchmark different array-based storage strategies for cells:

• arrayCellValuesSmallest(nCells int): Uses []SmallestCell.
• arrayCellValuesSmall(nCells int): Uses []SmallCell.
• arrayCellValuesSmaller(nCells int): Uses []SmallerCell.
• arrayCellValues(nCells int): Uses []LargeCell.
• arrayCellPointers(nCells int): Uses a slice of pointers []*LargeCell (simulates heap allocation per element).
• arrayCellPointersPreAllocated(nCells int): Uses a slice of pointers to elements within a pre-allocated arena (simulates arena allocation).