DOCSEARCHbyAlgolia
Submit Docs

Bentocache Documentation

repository·main·Indexed 20 days ago

https://github.com/julien-r44/bentocache

A high-performance, multi-tier caching solution for Node.js supporting L1 (in-memory) and L2 (distributed) caching with automatic synchronization via a Bus. It features resiliency patterns, cache stampede protection, and supports various drivers including Redis, Upstash, CloudflareKV, DynamoDB, Filesystem, and SQL databases (Knex, Kysely, Orchid). Includes official OpenTelemetry instrumentation via the @bentocache/otel package.

Tokens
37.2K
Snippets
128
Records
156
Agent score
70%

What's inside Bentocache

  1. What is Bentocache?

    main

    Bentocache is a robust, full-featured multi-tier caching library for Node.js applications. Unlike simple cache bridges, it provides advanced features for high-performance and resilient caching, including multi-layer support, cache stampede protection, and synchronization across multiple instances.

    Key capabilities include:

    • Multi-tier caching: Combines fast local in-memory (L1) with persistent distributed (L2) caches.
    • Resiliency: Features like grace periods and timeouts allow serving stale data if the backend or factory is slow/down.
    • Advanced Management: Supports Namespaces for grouping keys, Tagging for easy invalidation, and TTLs in human-readable formats.
    • Observability: Built-in event emission, OpenTelemetry instrumentation, and Prometheus integration.

  2. What is Bentocache and why use it?

    main

    Bentocache is a robust multi-tier caching solution for Node.js applications designed to combine high performance with flexibility. Unlike simple bridge libraries (like keyv or cache-manager) that primarily provide a unified API for different stores, Bentocache is a full-featured caching solution that includes advanced features like multi-layer caching, resiliency patterns, and cache stampede protection.

    Key benefits include:

    • Multi-tier caching: Combines fast in-memory (L1) access with persistent distributed (L2) storage.
    • Resiliency: Features like grace periods and timeouts allow serving stale data if the backend or store is slow or unavailable.
    • Advanced Management: Supports namespaces for grouping keys, tagging for easy invalidation, and cache stampede protection.

  3. Avoid key collisions between stores using the same backend

    main

    When multiple named stores use the same backend (e.g., the same Redis instance), you must prevent key collisions by using a prefix in the driver configuration.

    Using prefixes ensures that keys from different stores remain isolated. This isolation also applies to the .clear() method: calling .clear() on a specific store will only delete keys belonging to that store's prefix, leaving other stores untouched.

    const bento = new BentoCache({
  default: 'users',
  stores: {
    users: bentostore()
      .useL2Layer(redisDriver({ prefix: 'users' })),

    posts: bentostore()
      .useL2Layer(redisDriver({ prefix: 'posts' }))
  },
})

// Isolation in action:
bento.use('users').set({ key: 'foo', value: '2' })
bento.use('users').get({ key: 'foo' }) // '2'
bento.use('posts').get({ key: 'foo' }) // undefined

bento.use('posts').set({ key: 'foo', value: '1' })
bento.use('posts').get({ key: 'foo' }) // '1'

// Clearing only one store:
bento.use('users').clear()
bento.use('users').get({ key: 'foo' }) // undefined
bento.use('posts').get({ key: 'foo' }) // '1'

  4. Use soft timeouts to return stale data during grace periods

    main

    Soft timeouts (configured via the timeout option) allow you to set a maximum execution time for a factory when a grace period is active.

    If a request is made for a key that has expired but is still within its grace period, bentocache will attempt to run the factory. If the factory execution exceeds the specified timeout, bentocache will immediately return the expired (stale) entry from the cache to the user, while allowing the factory to continue executing in the background to refresh the cache for future requests.

    Note: If no entry is currently in its grace period in the cache, the soft timeout is ignored.

    const result = await bento.getOrSet({
  key: 'products',
  factory: () => Product.all(),
  ttl: '10m',
  grace: '6h',
  timeout: '200ms',
});

  5. Understanding Bentocache Benchmarks

    main

    The benchmarks in this repository are primarily used to detect performance regressions and are not intended as a definitive comparison of library superiority.

    Key takeaways from the benchmark methodology:

    • Single-tier vs. Multi-tier: While most libraries perform similarly on single-tier caches, Bentocache's primary performance advantage is observed in two-tier (multi-layer) cache configurations, a feature not supported by CacheManager.
    • Task Types:
      • mtier_get_key: Retrieves a key from the cache stack.
      • mtier_get_or_set: Retrieves a key or sets it if it doesn't exist (using large objects to test overhead).
    • Serialization: Benchmarks include scenarios both with and without object serialization in the memory cache to show the impact of data handling.

  6. Configure multi-tier caching with a synchronization bus

    main

    In multi-instance applications, you can synchronize in-memory (L1) caches across different instances using a bus (e.g., redisBusDriver). When an item is updated or deleted in one instance, the bus notifies other instances to update or invalidate their local L1 caches.

    If you are running on a single instance, a bus is not required.

    import { BentoCache, bentostore } from 'bentocache'
import { memoryDriver } from 'bentocache/drivers/memory'
import { redisDriver, redisBusDriver } from 'bentocache/drivers/redis'

const bento = new BentoCache({
  default: 'cache',

  stores: {
    cache: bentostore()
      .useL1Layer(memoryDriver({ maxSize: '10mb' }))
      .useL2Layer(redisDriver({ /* ... */ }))
      .useBus(redisBusDriver({ /* ... */ }))
  },
})

await bento.set({ key: 'user:42', value: { name: 'jul' } })

console.log(
  await bento.get({ key: 'user:42' })
)

  7. How two-level caching works in Bentocache

    main

    Bentocache supports a two-level caching architecture to maximize performance:

    1. L1 (Local Cache): An in-memory cache using an LRU (Least Recently Used) algorithm for extremely fast access.
    2. L2 (Distributed Cache): A shared cache (e.g., Redis) used if the data is not found in L1.
    3. Synchronization via Bus: In multi-instance environments, Bentocache uses a Bus (like Redis or RabbitMQ) to synchronize local in-memory caches across different instances, ensuring cache integrity.

    This pattern can provide responses between 2,000x and 5,000x faster than using a distributed cache alone, as accessing RAM is significantly faster than network calls to a distributed store.

  8. Stampede protection behavior in multi-instance applications

    main

    BentoCache uses in-memory locks for stampede protection. In multi-instance environments (such as applications running in cluster mode with PM2), each instance maintains its own independent lock system.

    While this means multiple instances might still trigger the factory function once each, the total number of redundant calls is significantly reduced. For example, if 10,000 requests are distributed across 10 application instances, BentoCache will reduce the load from 10,000 database queries to approximately 10 queries (one per instance).

  9. Synchronize L1 caches across instances using a Bus

    main

    In a multi-instance environment, an invalidation on one instance (e.g., deleting a key) won't automatically clear that key from the in-memory L1 caches of other instances. This leads to stale data.

    To prevent this, use .useBus() to add a synchronization layer. The bus sends messages to other instances notifying them to invalidate specific keys.

    Note: If your application runs on a single instance, a bus is not required; you only need L1 and L2 layers.

    import { BentoCache, bentostore } from 'bentocache'
import { memoryDriver } from 'bentocache/drivers/memory'
import { redisDriver, redisBusDriver } from 'bentocache/drivers/redis'

const redisConnection = { host: 'localhost', port: 6379 }

const bento = new BentoCache({
  default: 'multitier',
  stores: {
    multitier: bentostore()
      .useL1Layer(memoryDriver({ maxSize: '10mb' }))
      .useL2Layer(redisDriver({ connection: redisConnection }))
      .useBus(redisBusDriver({ connection: redisConnection }))
  }
})

  10. Protect against cache stampedes

    main
    Bentocache includes a built-in, transparent mechanism to prevent cache stampedes. When multiple concurrent requests attempt to access an expired key simultaneously, Bentocache ensures only one request triggers the factory function to refresh the data, while others wait or receive the cached value, preventing a surge of requests to your backend/database.

  11. How BentoCache protects against cache stampede

    main

    A cache stampede occurs when many clients request the same missing cache key simultaneously, causing the underlying data source (e.g., a database) to be overwhelmed by redundant requests.

    BentoCache prevents this by using an in-memory lock system. When a key is missing from the cache:

    1. BentoCache creates a lock for that specific key.
    2. The first request executes the factory function to fetch the data.
    3. Concurrent requests for the same key wait for the lock to be released.
    4. Once the first request completes and populates the cache, all waiting requests retrieve the value directly from the cache instead of re-executing the factory.

    This ensures that even with thousands of concurrent requests, the expensive factory function is only executed once per instance.

    router.get('/posts/:id', async (request) => {
  const { id } = request.params
  
  const post = await bento.getOrSet({
    key: `post:${id}`, 
    factory: () => getPostFromDb(id),
    ttl: '1h',
  })
  
  return post
})

  12. What are grace periods in BentoCache?

    main

    A grace period is a duration during which stale cache entries can still be served if the underlying data source (the factory function) fails to return a value.

    When a cache entry's TTL (Time To Live) expires, it becomes 'stale'. Normally, a request for a stale entry triggers a call to the factory to refresh the cache. If that factory call fails (e.g., due to a database outage), BentoCache will use the grace period to serve the expired data instead of returning an error to the user. This enhances system resilience and improves user experience during downtimes.