bigcache

repository·main·Indexed 27 days ago

https://github.com/allegro/bigcache

A fast, concurrent, evicting in-memory cache for Go designed to handle a large number of entries while minimizing GC overhead by storing entries in byte slices. It features configurable sharding, eviction windows (LifeWindow and CleanWindow), and memory limits via the bigcache.Config struct. The library includes a RESTful HTTP Server API for managing cache entries and retrieving statistics, as well as support for custom hashers and removal callbacks.

Tokens
3.7K
Snippets
4
Records
25
Agent score
93%

What's inside bigcache

  1. Operational Notes for BigCache HTTP Server

    main

    When deploying the BigCache HTTP Server, keep the following operational constraints in mind:

    • Security: There is currently no SSL/TLS support and no authentication mechanism.
    • Persistence: Statistics from the stats API are not persistent and reset on restart.
    • Clustering: There is no built-in replication or clustering support.
    • Cache Clearing: The fastest way to clear the entire cache is to restart the process; initialization typically takes less than a second.
  2. Initialize BigCache with simple configuration

    main

    For basic use cases, you can initialize BigCache using bigcache.New combined with bigcache.DefaultConfig. This approach requires a context.Context and a duration representing the entry lifetime.

    Note that BigCache operates on byte slices, so you must handle (de)serialization of your data before interacting with the cache.

    import (
    	"context"
    	"fmt"
    	"time"
    	"github.com/allegro/bigcache/v3"
    )
    
    cache, _ := bigcache.New(context.Background(), bigcache.DefaultConfig(10 * time.Minute))
    
    cache.Set("my-unique-key", []byte("value"))
    
    entry, _ := cache.Get("my-unique-key")
    fmt.Println(string(entry))
  3. Configure BigCache with custom settings

    main

    If you can predict your cache load in advance, use a custom bigcache.Config to avoid additional memory allocations. This allows fine-grained control over sharding, eviction windows, and memory limits.

    import (
    	"context"
    	"fmt"
    	"log"
    	"time"
    
    	"github.com/allegro/bigcache/v3"
    )
    
    config := bigcache.Config {
    		// number of shards (must be a power of 2)
    		Shards: 1024,
    
    		// time after which entry can be evicted
    		LifeWindow: 10 * time.Minute,
    
    		// Interval between removing expired entries (clean up).
    		// If set to <= 0 then no action is performed.
    		// Setting to < 1 second is counterproductive — bigcache has a one second resolution.
    		CleanWindow: 5 * time.Minute,
    
    		// rps * lifeWindow, used only in initial memory allocation
    		MaxEntriesInWindow: 1000 * 10 * 60,
    
    		// max entry size in bytes, used only in initial memory allocation
    		MaxEntrySize: 500,
    
    		// prints information about additional memory allocation
    		Verbose: true,
    
    		// cache will not allocate more memory than this limit, value in MB
    		// if value is reached then the oldest entries can be overridden for the new ones
    		// 0 value means no size limit
    		HardMaxCacheSize: 8192,
    
    		// callback fired when the oldest entry is removed because of its expiration time or no space left
    		// for the new entry, or because delete was called. A bitmask representing the reason will be returned.
    		// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
    		OnRemove: nil,
    
    		// OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left
    		// for the new entry, or because delete was called. A constant representing the reason will be passed through.
    		// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
    		// Ignored if OnRemove is specified.
    		OnRemoveWithReason: nil,
    }
    
    cache, initErr := bigcache.New(context.Background(), config)
    if initErr != nil {
    	log.Fatal(initErr)
    }
    
    cache.Set("my-unique-key", []byte("value"))
    
    if entry, err := cache.Get("my-unique-key"); err == nil {
    	fmt.Println(string(entry))
    }
  4. Understand LifeWindow and CleanWindow behavior

    main

    BigCache uses two distinct time windows for managing entry expiration:

    1. LifeWindow: The duration after which an entry is considered "dead" but is not yet physically deleted from the cache.
    2. CleanWindow: The interval at which the cache performs a cleanup. During this window, all "dead" entries are deleted, but entries that are still within their LifeWindow are preserved.
  5. Use the BigCache HTTP Server API

    main

    The BigCache HTTP Server provides a RESTful API for managing cached data and retrieving statistics. It accepts any content type, making it suitable for caching text, images, or software artifacts.

    Cache API

    Use these endpoints to manage individual cache entries:

    • GET /api/v1/cache/{key}: Retrieve the value associated with the key.
    • PUT /api/v1/cache/{key}: Store a value at the specified key.
    • DELETE /api/v1/cache/{key}: Remove the entry for the specified key.

    Stats API

    • GET /api/v1/stats: Returns hit and miss statistics. Note that statistics are not persistent and reset whenever the server is restarted.
  6. Configure BigCache using the Config struct

    main

    The Config struct defines the behavior and resource limits of a BigCache instance. Use this struct to tune performance, memory usage, and eviction policies.

    Key Configuration Fields

    FieldDescription
    ShardsNumber of cache shards. Must be a power of two.
    LifeWindowThe duration after which an entry can be evicted.
    CleanWindowInterval between removing expired entries. Must be $\ge$ 1 second. If $\le$ 0, no cleanup is performed.
    MaxEntriesInWindowMax number of entries in the life window. Used to calculate initial shard size to prevent reallocations.
    MaxEntrySizeMax size of an entry in bytes. Used to calculate initial shard size.
    HardMaxCacheSizeLimit for BytesQueue size in MB. If reached, oldest entries are overridden. 0 means unlimited.
    StatsEnabledIf true, tracks how many times a resource was requested.
    VerboseIf true, prints information about new memory allocations.
    HasherCustom Hasher to map string keys to uint64. Defaults to fnv64.
    OnRemoveCallback for when an entry is removed (via expiration, no space, or manual delete).
    OnRemoveWithMetadataCallback providing Metadata about the removed entry. Overrides OnRemove.
    OnRemoveWithReasonCallback providing a RemoveReason for the eviction. Overrides OnRemove and OnRemoveWithMetadata.
    LoggerLogging interface used with Verbose. Defaults to DefaultLogger().
  7. Configure bigcache.Config options

    main

    The bigcache.Config struct provides the following configuration options:

    • Shards: Number of shards (must be a power of 2).
    • LifeWindow: Time after which an entry can be considered dead but not yet deleted.
    • CleanWindow: Interval between removing expired entries. If set to $\le 0$, no action is performed. Setting to $< 1$ second is counterproductive as BigCache has a one-second resolution.
    • MaxEntriesInWindow: Used only in initial memory allocation (calculated as $rps \times lifeWindow$).
    • MaxEntrySize: Max entry size in bytes, used only in initial memory allocation.
    • Verbose: If true, prints information about additional memory allocation.
    • HardMaxCacheSize: Maximum memory limit in MB. If reached, oldest entries are overridden. A value of 0 means no size limit.
    • OnRemove: Callback fired when the oldest entry is removed (due to expiration, lack of space, or manual delete). Returns a bitmask representing the reason.
    • OnRemoveWithReason: Callback fired when the oldest entry is removed. Passes a constant representing the reason. This is ignored if OnRemove is specified.
  8. Configure the BigCache HTTP Server via CLI

    main

    The server can be configured using several command-line flags during startup.

    Available Flags

    FlagDescription
    -lifetimeLifetime of each cache object (default: 10m0s)
    -logfileLocation of the logfile
    -maxMaximum amount of data in the cache in MB (default: 8192)
    -maxInWindowUsed only in initial memory allocation (default: 600000)
    -maxShardEntrySizeMaximum size of each object stored in a shard. Used only in initial memory allocation (default: 500)
    -portThe port to listen on (default: 9090)
    -shardsNumber of shards for the cache (default: 1024)
    -vEnable verbose logging
    -versionPrint server version