DOCSEARCHbyAlgolia
Submit Docs

cacheable

repository·main·Indexed 24 days ago

https://github.com/jaredwray/cacheable

A suite of scalable Node.js caching packages built on Keyv. It includes cache-manager for multi-layer (tiered) caching with support for background refreshes and wrap functions, as well as cacheable-request for RFC 7234 compliant caching of native Node.js HTTP/HTTPS requests.

Tokens
63.1K
Snippets
148
Records
299
Agent score
84%

What's inside cacheable

  1. Overview of Cacheable packages

    main

    Cacheable is a collection of Node.js caching packages built on top of Keyv. Depending on your use case, you can choose from several specialized packages:

    • cacheable: A next-generation caching framework featuring layer 1 / layer 2 caching.
    • cache-manager: A robust cache manager used in services like NestJS, supporting features like wrap.
    • cacheable-request: Adds RFC-compliant cache support to native HTTP requests.
    • flat-cache: Fast in-memory caching with file store persistence.
    • file-entry-cache: A lightweight cache for file metadata, useful for tracking file changes.
    • @cacheable/node-cache: A maintained replacement for the node-cache package.
    • @cacheable/memory: In-memory caching with LRU (Least Recently Used) support.
    • @cacheable/utils: Utility functions including hashing, shorthand time, and memoize.

  2. Enable non-blocking secondary storage

    main

    By default, cacheable waits for the secondary (layer 2) store to respond during data operations. If you want faster response times and do not need to wait for the secondary store to confirm setting data, deleting data, or clearing data, set the nonBlocking property to true in the Cacheable options.

    In nonBlocking mode, the following behaviors occur:

    • set: Updates the primary storage immediately, then updates the secondary in the background.
    • get, getMany, getRaw, getManyRaw: Checks the primary storage immediately. If the value is not found, it looks for the value in the secondary storage in the background and updates the primary storage if found.
    const cache = new Cacheable({ secondary, nonBlocking: true });

  3. Use tag-based invalidation with Cacheable

    main

    You can associate cache entries with tags to invalidate groups of entries simultaneously. This is ideal for scenarios where one entity (e.g., a user or product) is referenced by multiple cache keys.

    Key Concepts:

    • Lazy Invalidation: invalidateTag uses a constant-time model by bumping a version counter for the tag.
    • Freshness Check: On get or getMany, the cache compares the entry's tag version snapshot against the live version. If they mismatch, the entry is treated as a miss and purged from both primary and secondary stores.
    • Distributed Invalidation: If using a shared secondary store (like Redis), an invalidation on one instance is visible to all instances.
    • Enabling Tags: The tag service is disabled by default. You must enable it via the constructor option { tags: true } or by setting cache.tags.enabled = true.

    Warning: To ensure consistent behavior in distributed environments, enable the tag service on every instance that shares the store (both writers and readers).

    import { Cacheable } from 'cacheable';

const cache = new Cacheable({ tags: true });

await cache.set('page:/products', html, { ttl: '10m', tags: ['entity:42', 'collection:products'] });
await cache.set('page:/products/42', detailHtml, { ttl: '10m', tags: ['entity:42'] });

// entity 42 changed - purge everything that referenced it
await cache.tags.invalidateTag('entity:42');

await cache.get('page:/products'); // undefined
await cache.get('page:/products/42'); // undefined

  4. Enable LRU (Least Recently Used) eviction

    main

    Enable the LRU feature by setting the lruSize property in the options. This limits the total number of keys in the cache. When the limit is reached, the least recently used entries are evicted.

    Constraints:

    • lruSize is capped at 16,777,216 (2^24) keys. Values above this are rejected and an error event is emitted.
    • Setting lruSize: 0 disables the LRU feature and removes the key limit (allowing you to exceed the single Map limit via hashing).
    import { CacheableMemory } from 'cacheable';
const cache = new CacheableMemory({ lruSize: 1 }); // sets the LRU cache size to 1 key
cache.set('key1', 'value1');
cache.set('key2', 'value2');
const value1 = cache.get('key1');
console.log(value1); // undefined if the cache is full and key1 is the least recently used

  5. Enable background cache refreshing with `refreshThreshold`

    main

    You can prevent cache misses by refreshing expiring keys in the background. This is enabled by setting a refreshThreshold (in seconds) in the configuration.

    How it works:

    1. When a value is retrieved via wrap, the system checks its remaining TTL.
    2. If the remaining TTL is less than the refreshThreshold, a background worker is spawned to re-fetch the data using the same rules as a standard wrap call.
    3. The system returns the old value to the user immediately, ensuring no latency penalty.
    4. In multiCaching, the refresh happens in the highest priority store where the key was found, and the new value is then propagated to all other stores.

    Requirements & Limitations:

    • The underlying cache store must implement a ttl() method.
    • Background refresh currently does not support wrap calls with multiple keys.
    • If the threshold is too low or the worker function is too slow, a race condition might occur where the key expires before the update completes.
    var redisCache = cacheManager.caching({
  store: redisStore,
  refreshThreshold: 3, // Refresh if TTL < 3 seconds
  isCacheableValue: isCacheableValue,
});

  6. Use Simple Caching Mode

    main

    Set httpCachePolicy: false to enable a simple key-based caching mode that ignores HTTP cache directives.

    In this mode:

    • Every successful GET response is cached regardless of headers.
    • The library uses the default TTL provided in the Cacheable instance configuration.
    • Cached entries are never revalidated.
    • Concurrent identical misses are coalesced so the origin is only hit once.

    Note: Error responses (4xx / 5xx) are always returned to the caller but are never cached.

    import { CacheableNet } from '@cacheable/net';

const net = new CacheableNet({
  httpCachePolicy: false,
  cache: { ttl: '5m' } // every cached GET lives for 5 minutes
});

const { data } = await net.get('https://api.example.com/data');

  7. Refresh cache keys in the background

    main
    To prevent latency spikes when a cache key expires, you can use the refreshThreshold option. When using wrap(), if the remaining TTL of a retrieved value is less than the refreshThreshold, the system triggers an asynchronous background refresh using the same logic as a standard fetch. The system returns the existing (old) value immediately while the refresh happens in the background.

  8. Track cache metrics with the Stats class

    main

    The Stats class provides an event-driven way to track cache performance (hits, misses, rates, etc.).

    Key Features:

    • Opt-in: Tracking is disabled by default (enabled: false) to ensure zero overhead when not in use.
    • Manual Updates: Use increment() / decrement() or named helpers like incrementHits().
    • Automatic Updates: Can be subscribed to an EventEmitter to update counters based on cache events.
    • Snapshots: Use toJSON() to get a plain object of all current metrics for logging or external monitoring.
    import { Stats } from '@cacheable/utils';

const stats = new Stats({ enabled: true });

stats.incrementHits();
stats.incrementMisses();
stats.incrementGets();

console.log(stats.hits);    // 1
console.log(stats.misses);  // 1
console.log(stats.hitRate); // 0.5

// Exporting metrics
console.log(stats.toJSON());

  9. Iterate over cache stores

    main

    The Cacheable class exposes primary and secondary as Keyv instances. You can iterate over entries using the iterator() async generator, but you must feature-check for its existence because not all stores support it.

    Supported Iterators

    Keyv provides iterator() for:

    • A plain new Keyv() (Map-backed)
    • @keyv/redis, @keyv/valkey, @keyv/mongo, @keyv/sqlite, @keyv/postgres, @keyv/mysql, and @keyv/etcd.

    Iterating the default in-memory primary

    The default primary store (@cacheable/memory) does not support iterator(). To walk it, you must access the underlying CacheableMemory store via cache.primary.store and use its items property. Each item contains { key, value, expires }, where item.value.value is the decoded data.

    Performance Warning

    Iteration can be expensive on large datasets (e.g., Redis SCAN). Avoid using it on hot paths.

    // Example: Iterating a secondary store that supports it
if (cache.secondary?.iterator) {
    for await (const [key, value] of cache.secondary.iterator()) {
        console.log(`${key}:`, value);
    }
}

// Example: Iterating the default in-memory primary
import { Cacheable, KeyvCacheableMemory } from 'cacheable';
const cache = new Cacheable();
const memory = (cache.primary.store as KeyvCacheableMemory).store;

for (const item of memory.items) {
    console.log(`${item.key}:`, item.value.value);
}

  10. How tiered (multi-store) caching works

    main

    A tiered cache strategy allows you to use multiple cache stores simultaneously (e.g., an in-memory cache backed by Redis).

    How it works:

    1. Priority: Data is fetched from the highest priority cache(s) first.
    2. Transparency: The application interacts with a single interface, while cache-manager handles the movement of data between layers.
    3. Multi-get merging: When using mget across multiple stores, cache-manager traverses the tiers from highest to lowest priority and merges the values found at each level into the final result.

    Use Case: Use a short-lived, small in-memory cache for extremely high-traffic keys to prevent hitting a primary distributed cache (like Redis) for every single request.

  11. Configure CacheableMemory Store Hashing

    main

    To scale past the 16,777,216 (2^24) keys limit of a single JavaScript Map, CacheableMemory uses multiple Map objects. It hashes keys to distribute them across these stores using the storeHashAlgorithm option.

    Supported Algorithms:

    • DJB2 (default)
    • FNV1
    • MURMER
    • CRC32

    Note: Cryptographic algorithms like SHA-256 are not recommended due to performance overhead. You can also provide a custom function that returns a number between 0 and storeSize - 1.

    import { CacheableMemory, HashAlgorithm } from '@cacheable/memory';

// Using DJB2 (default)
const cache = new CacheableMemory({ storeHashAlgorithm: HashAlgorithm.DJB2 });

// Using FNV1 for performance
const cache2 = new CacheableMemory({ storeHashAlgorithm: HashAlgorithm.FNV1 });

// Custom hashing function
const customHash = (key, storeHashSize) => {
  return key.length % storeHashSize;
};
const cache3 = new CacheableMemory({ storeHashAlgorithm: customHash, storeSize: 32 });

  12. Understand Storage Tiering (Layer 1 and Layer 2)

    main

    Cacheable implements a two-tier caching engine:

    • Primary Store (Layer 1): Fast, local, or in-memory storage.
    • Secondary Store (Layer 2): More persistent, shared storage (e.g., Redis).

    Default Operations:

    • Setting Data: Sets the value in both primary and secondary stores.
    • Getting Data: Checks primary first. If missing, it fetches from secondary and populates the primary store.
    • Deleting Data: Deletes from both stores simultaneously.
    • Clearing Data: Clears both stores simultaneously.

    TTL Propagation: When a value is retrieved from the secondary store and set into the primary, it inherits the remaining TTL from the secondary store, subject to the primary store's own TTL constraints.