DOCSEARCHbyAlgolia
Submit Docs

HOT (Hot Object Tracker)

repository·main·Indexed 42 days ago

https://github.com/samber/hot

A high-performance, type-safe in-memory caching library for Go using generics. It supports multiple eviction policies (LRU, LFU, S3FIFO, TinyLFU, W-TinyLFU, ARC, 2Q, FIFO), TTL with jitter, and stale-while-revalidate patterns. Features include sharding to reduce lock contention, batch operations (SetMany, GetMany) for improved throughput, and a 'Missing Cache' mode to prevent cache stampedes. Includes integrations for Prometheus metrics and PostgreSQL cache-aside patterns.

Tokens
17.4K
Snippets
58
Records
86
Agent score
54%

What's inside HOT

  1. Core Features of the HOT Simple Cache

    main

    The simple LRU cache implementation demonstrates several key capabilities of the HOT library:

    • LRU Eviction: Automatically evicts the Least Recently Used items when the cache reaches its capacity (e.g., 1000 items).
    • TTL Support: Allows items to expire automatically after a specified duration (e.g., 5 minutes).
    • Batch Operations: Provides more efficient ways to perform multiple operations at once compared to individual calls.
    • Cache Statistics: Enables monitoring of cache utilization and performance metrics.

  2. Choose the right eviction policy

    main

    HOT supports several eviction algorithms. Choosing the right one depends on your workload:

    • W-TinyLFU (Default): Best all-around performer for most scenarios.
    • TinyLFU / LFU: Best for frequency-heavy workloads.
    • W-TinyLFU / ARC: Best for workloads with changing access patterns.
    • LRU: Best for recency-heavy workloads.
    • S3-FIFO: Best for stable, throughput-critical workloads.

  3. Use different eviction policies

    main

    HOT provides several eviction policies to manage memory by removing items when capacity is reached. Each policy implements pkg/base.InMemoryCache[K, V].

    Note: Low-level eviction policies (from pkg/lru, pkg/lfu, etc.) are not thread-safe by default. If you use them directly, you must wrap them in pkg/safe.SafeInMemoryCache to ensure concurrent access safety.

    // Example of creating a raw LRU cache (not thread-safe)
cache := lru.NewLRUCache[string, *User](100_000)

// Example of making it thread-safe
import (
    "github.com/samber/hot/pkg/lfu"
    "github.com/samber/hot/pkg/safe"
)

cache := safe.NewSafeInMemoryCache(
    lru.NewLRUCache[string, *User](100_000),
)

  4. Choose the right eviction policy for your workload

    main

    HOT supports several eviction algorithms. Choosing the right one depends on your access patterns:

    • W-TinyLFU (Default): The most robust general-purpose choice; adapts between recency and frequency.
    • TinyLFU: Best for skewed workloads with many rare items; filters out 'one-hit' items.
    • S3-FIFO: Ideal for high-throughput systems with stable workloads.
    • ARC: Best for workloads that change patterns over time; requires no tuning.
    • LFU: Best for stable popularity (hot keys).
    • LRU: Best for recency-based workloads.
    • 2Q: Better than LRU for datasets larger than the cache size.
    • FIFO: Use only when simplicity is the primary requirement.

  5. Understand HOT thread safety guarantees

    main

    The HOT cache is designed for high-performance concurrent environments and provides the following guarantees:

    • Atomic Operations: All cache operations are thread-safe.
    • Lock-Free Reads: Optimized to allow high read concurrency without blocking.
    • Minimal Lock Contention: Uses efficient locking strategies to reduce overhead during simultaneous access.
    • Consistent State: Ensures no data races or state corruption occur during concurrent access.

  6. How the HOT architecture works

    main

    HOT uses a layered architecture where each layer implements the pkg/base.InMemoryCache[K, V] interface. This allows you to compose different functionalities like sharding, metrics, thread safety, and eviction policies.

    While you can use low-level layers individually, it is highly recommended to use hot.HotCache[K, V] as the high-level, feature-complete entry point.

    Layers typically stack in this order:

    1. High-level: hot.HotCache (Features, Loaders, TTL)
    2. Sharding: pkg/sharded.ShardedInMemoryCache (Reduces lock contention)
    3. Metrics: pkg/metrics.InstrumentedCache (Collection)
    4. Safety: pkg/safe.SafeInMemoryCache (Thread safety)
    5. Eviction: Various policies like LRUCache, LFUCache, FIFOCache, etc.

  7. Leverage HOT performance features

    main

    HOT is designed for high-concurrency Go applications using several architectural optimizations:

    • Sharding: The cache is partitioned into shards to reduce lock contention during high-concurrency access.
    • Batch Operations: Provides efficient bulk operations to improve throughput.
    • Copy-on-Read/Write: An optional feature that provides value copying to ensure thread safety when dealing with mutable data.

  8. Understand Cache Modes: Main vs Missing Cache

    main

    HOT utilizes two distinct cache modes to optimize performance and prevent common issues:

    • Main Cache: The primary storage for your actual data values.
    • Missing Cache: Stores negative results (the fact that a key does not exist). This is used to prevent "cache stampedes" or repeated expensive lookups for keys that are known to be missing.

  9. Use Jitter to prevent cache stampedes

    main
    When configuring a cache, you can apply jitter to the TTL. Jitter adds a random offset to the expiration time of items. This ensures that items stored at roughly the same time do not all expire at the exact same moment, which helps smooth out resource consumption and prevents simultaneous backend reloads.

  10. Choose an eviction policy for your cache

    main

    HOT supports several eviction algorithms to manage how items are removed when the cache reaches capacity. Choose the one that best fits your access patterns:

    • LRU (Least Recently Used): Evicts items that haven't been accessed recently.
    • LFU (Least Frequently Used): Evicts items with the lowest access frequency.
    • S3FIFO (Segmented 3-Level FIFO): An extension of FIFO with three segments to reduce cache pollution and handle varying access patterns.
    • TinyLFU: Uses an approximate frequency histogram to track popularity and decide eviction efficiently.
    • W-TinyLFU (Windowed TinyLFU): Combines a small "window" cache with TinyLFU to handle both recency and frequency.
    • ARC (Adaptive Replacement Cache): Automatically adapts between LRU and LFU.
    • 2Q (Two Queue): Uses two queues to separate frequently and infrequently accessed items.
    • FIFO (First In, First Out): Evicts items in the order they were added.

  11. Install HOT cache

    main

    Install the HOT caching library using go get:

    go get github.com/samber/hot

    Note: This library is currently in v0 and follows SemVer strictly. Breaking changes may occur before v1.0.0.

  12. Integrate HOT with PostgreSQL

    main

    This example demonstrates how to implement a cache-aside pattern using HOT and a PostgreSQL database. The integration involves creating a database manager that uses HOT as a caching layer, allowing for automatic data loading from the database when a cache miss occurs, and automatic cache invalidation during CRUD operations.

    Key patterns demonstrated:

    • Cache-Aside Pattern: Attempt to load data from the cache; if missing, fall back to the database.
    • Automatic Loading: The cache automatically fetches missing data from the DB using a loader.
    • Cache Invalidation: Update and delete operations on the database trigger invalidation in the cache to ensure data consistency.
    • Error Handling: Graceful management of database connection or query errors.
    # Install PostgreSQL driver
go mod download

# Set up database connection
export DATABASE_URL="postgres://user:password@localhost/dbname?sslmode=disable"

# Run the example
go run postgresql-cache.go