GitHub Actions Cache Server

The system uses two types of expiring leases to ensure data integrity during concurrent downloads and merge operations:

1. Part Reader Leases: Protect concurrent readers of individual Parts. These prevent Parts from being deleted while a worker is actively reading them.
2. Storage Reader Leases: Protect the overall Storage Location. These prevent the entire Storage Location from being deleted while active downloads are in progress.

Lease Lifecycle and Expiration

• Renewal: Both lease types automatically renew every 30 seconds while active.
• Expiration: Standard leases expire after two minutes of inactivity.
• Direct-Download Exception: Leases for direct downloads expire alongside their associated signed URLs.

Safety Guarantees

• Atomic Acquisition: A reader acquires its lease within the same locked transaction used to read the Storage Location's merge state. This ensures the reader's scope is chosen atomically.
• Deletion Protection:
  • Parts are only deleted after a merge completes and no active Part Reader Leases remain.
  • A Storage Location is only deleted when no active Storage Reader Leases remain.
• Race Condition Prevention: Because lease acquisition is atomic with the state read, a download arriving before a merge will take a Part Reader Lease (blocking part deletion), while a download arriving after the merge will take a Storage Reader Lease (protecting the merged representation). This eliminates the window where a reader could attempt to access parts that are being deleted.

To prevent hard build failures in clients like BuildKit's gha cache importer, the server performs storage validation during the cache lookup process. If a cache entry exists in the database but the underlying object storage has been modified externally (e.g., via bucket lifecycle rules, manual wipes, or out-of-sync database restores), the server identifies this as a 'Dangling Cache Entry'.

Validation Logic:

• Merged entries: Validated via a HEAD request on the merged object.
• Unmerged entries: Validated via a HEAD request on the first part (parts/0).
• Dangling entries: If the HEAD request fails, the entry is considered dangling.

Self-Healing Mechanism: When a dangling entry is detected, the server automatically:

1. Deletes the dangling cache_entries row.
2. Reaps the associated storage_location (which is otherwise invisible to background Orphaned Storage sweeps).
3. Re-runs the matching logic to see if a valid candidate exists under a different restore key.
4. If no valid match is found, it returns { ok: false } to signal a clean cache miss, allowing the client to proceed without a hard failure.

The server manages storage using a Storage Budget and Capacity-based Eviction to prevent disk exhaustion.

• Storage Budget: The maximum amount of finalized cache payloads allowed. For filesystem backends, this defaults to 90% of the Filesystem Capacity (the total capacity of the mounted volume). For object-storage backends, there is no budget unless an explicit maximum is configured.
• Capacity-based Eviction: When an upload completes, the server may trigger an eviction pass to bring usage back down to 90% of the budget.
• Eviction Ordering: Eviction is ordered by Cache Recency. Cache Recency is determined by the most recent Cache Access (authorization to retrieve a payload), falling back to the time the entry was last saved or replaced.

Note on Configuration: When configuring limits, use the terms Storage Budget and Filesystem Capacity. Avoid using Storage limit, disk limit, or Cache directory size to prevent ambiguity.

Capacity-based eviction is a mechanism that manages storage usage by deleting old cache entries when a predefined storage budget is exceeded.

Key behaviors:

• Trigger: Eviction runs only after an upload completes and only if the current usage exceeds the configured Storage Budget.
• Eviction Order: Entries are deleted in Cache Recency order (the most recently accessed entries are preserved). If recency data is unavailable, the system falls back to the Cache Entry's save time.
• Eviction Target: The system deletes entries until usage is at most 90% of the budget.
• Safety: Active Storage Reader Leases prevent the deletion of files currently being read.
• Concurrency: Because there are no global capacity locks or pre-upload reservations, concurrent uploads may temporarily exceed the budget. Operators should ensure there is sufficient headroom for in-progress uploads and high concurrency.

To use the cache server effectively, you must understand how it manages data, storage, and eviction. The server distinguishes between data currently being received (Upload) and data that is finalized and available for workflows (Cache Entry).

Key lifecycle concepts include:

• Merge: The process of consolidating segments of data (Parts) into a single consolidated representation. This is protected by a Merge Lease to ensure only one worker performs the operation.
• Cache Hit vs. Miss: A Cache Hit occurs when a request matches an existing Cache Entry (via exact key or restore-key prefix) and passes validation. A Cache Miss occurs when no usable entry is found, including cases where only a Dangling Cache Entry (an entry pointing to non-existent storage) was available.
• Results Passthrough: If the server cannot handle a specific Results request, it performs a Results Passthrough to the Default Results Origin (configured via DEFAULT_ACTIONS_RESULTS_URL).

The server uses the database as the authoritative source of truth for cache entries. If a discrepancy exists between the database and the physical storage (filesystem root or gh-actions-cache/ object-store prefix), the database state takes precedence.

Key Behaviors:

• Atomic Deletions: Database state changes are committed before physical deletions occur. If a physical deletion fails, the system treats the resulting data as Orphaned Storage for later reconciliation, rather than leaving a broken cache entry.
• Automatic Cleanup: Data that is not referenced by any upload or storage location is eligible for deletion. To prevent accidental loss of new data, a grace period is applied. Data can only be deleted once its newest object is older than the configured grace period (which defaults to 24 hours).
• Database Restores: Be aware that restoring an older database version while keeping newer physical storage may result in the deletion of data. This happens because the older database does not reference the newer physical objects, causing them to be treated as untracked and subject to the automatic cleanup policy.

This design prioritizes bounded storage usage and automatic recovery from interrupted cleanup processes over the indefinite preservation of untracked data.

To use Google Cloud Storage (GCS) as the backend for the cache server, you can initialize the GcsAdapter using the fromEnv static method. This method requires specific environment variables to be set.

Required environment variables:

• STORAGE_DRIVER: Must be set to gcs.
• STORAGE_GCS_BUCKET: The name of your GCS bucket.
• STORAGE_GCS_SERVICE_ACCOUNT_KEY: Path to your GCS service account key file.
• STORAGE_GCS_ENDPOINT: The API endpoint for GCS (if using a custom endpoint or emulator).

Note: The adapter automatically uses the prefix gh-actions-cache for all objects stored in the bucket to prevent namespace collisions.

// The adapter is initialized via environment variables
// STORAGE_DRIVER=gcs
// STORAGE_GCS_BUCKET=my-cache-bucket
// STORAGE_GCS_SERVICE_ACCOUNT_KEY=/path/to/key.json
// STORAGE_GCS_ENDPOINT=https://storage.googleapis.com

const adapter = await GcsAdapter.fromEnv(env);