Quick Cache

Quick Cache is a lightweight, high-performance concurrent cache optimized for low overhead. It is designed for use cases where cache access times and overhead are significant costs.

Key features include:

• S3-FIFO Policy: Scan-resistant and high hit rate caching policy.
• Concurrency: Scales well with threads; supports atomic operations via get_or_insert, get_value_or_guard, and their async counterparts (get_or_insert_async, get_value_or_guard_async).
• Customizable Weight: Users can define weight per item.
• Non-blocking: Methods return immediately on lock contention.
• Entry API: Closure-based entry API for atomic inspect-and-act patterns (keep, remove, replace).
• Low Overhead: Small dependency tree and no background threads.

Note: If you require features like Time-to-Live (TTL) or event listeners, consider using the Moka crate instead.

You can control how the cache calculates the weight of items by implementing the Weighter trait. This is useful when items have varying sizes (e.g., string length) and you want to limit the cache based on total weight rather than item count.

Use Cache::with_weighter(max_capacity, max_weight, weighter) to initialize the cache.

Warning: Be cautious about zero weights, as they can lead to unexpected behavior in capacity management.

use quick_cache::{Weighter, sync::Cache};

#[derive(Clone)]
struct StringWeighter;

impl Weighter<u64, String> for StringWeighter {
    fn weight(&self, _key: &u64, val: &String) -> u64 {
        // Be cautious about zero weights!
        val.len() as u64
    }
}

fn main() {
    let cache = Cache::with_weighter(100, 100_000, StringWeighter);
    cache.insert(1, "1".to_string());
    cache.insert(54, "54".to_string());
    cache.insert(1000, "1000".to_string());
    assert_eq!(cache.get(&1000).unwrap(), "1000");
}

The Cache struct provides a single-threaded, non-concurrent cache implementation. You can initialize it using several methods depending on how you want to manage capacity and weight.

• new(items_capacity: usize): Creates a cache that holds approximately items_capacity items.
• with_weighter(estimated_items_capacity: usize, weight_capacity: u64, weighter: We): Creates a cache with a custom Weighter to track weight instead of just item count.
• with(...): A low-level constructor for specifying capacity, weight, weighter, hash builder, and lifecycle.
• with_options(options: Options, ...): Constructs a cache using an Options object built via OptionsBuilder.

use quick_cache::unsync::{Cache, OptionsBuilder, UnitWeighter, DefaultHashBuilder, DefaultLifecycle};

// Example: Using with_options
Cache::<(String, u64), String>::with_options(
  OptionsBuilder::new()
    .estimated_items_capacity(10000)
    .weight_capacity(10000)
    .build()
    .unwrap(),
    UnitWeighter,
    DefaultHashBuilder::default(),
    DefaultLifecycle::default(),
);

You can create a new Cache using several constructors depending on your requirements for capacity and weighting:

• new(items_capacity: usize): Creates a cache with an approximate item capacity.
• with_weighter(estimated_items_capacity: usize, weight_capacity: u64, weighter: We): Creates a cache with a specific weight capacity and a custom Weighter.
• with(estimated_items_capacity: usize, weight_capacity: u64, weighter: We, hash_builder: B, lifecycle: L): A full constructor for specifying capacity, weighting, hashing, and lifecycle behavior.
• with_options(options: Options, weighter: We, hash_builder: B, lifecycle: L): Constructs a cache using an Options object from OptionsBuilder.

Note on Values: Cache values are cloned when fetched. To avoid expensive clones, wrap your values in Arc<_>. For interior mutability, use Arc<Mutex<_>> or Arc<RwLock<_>>.

use quick_cache::sync::{Cache, DefaultLifecycle, OptionsBuilder, UnitWeighter, DefaultHashBuilder};

// Example using with_options
let cache = Cache::<(String, u64), String>::with_options(
  OptionsBuilder::new()
    .estimated_items_capacity(10000)
    .weight_capacity(10000)
    .build()
    .unwrap(),
    UnitWeighter,
    DefaultHashBuilder::default(),
    DefaultLifecycle::default(),
);

Use OptionsBuilder to configure the behavior and capacity of a quick_cache instance. The builder must be used to set weight_capacity and estimated_items_capacity before calling .build(), otherwise an error will be returned. The resulting Options struct is then passed to Cache::with_options.

Required Fields

• weight_capacity(u64): The maximum weight the cache can hold.
• estimated_items_capacity(usize): The expected number of items in the cache (roughly weight_capacity / average item weight). This helps the cache estimate shard sizes and ghost key tracking space.

Optional Fields

• shards(usize): Number of internal shards. Higher numbers reduce contention in multi-threaded environments. Defaults to number of detected cores * 4. The value is internally adjusted to the next power of two.
• hot_allocation(f64): Percentage [0.0, 1.0] of cache space reserved for "hot" items. Defaults to 0.97. Lower this if your workload favors recency over frequency.
• ghost_allocation(f64): Percentage [0.0, 1.0] of estimated_items_capacity used to track recently evicted keys ("ghost keys"). Defaults to 0.5.

use quick_cache::{sync::{Cache, DefaultLifecycle}, OptionsBuilder, UnitWeighter, DefaultHashBuilder};

Cache::<String, String>::with_options(
  OptionsBuilder::new()
    .estimated_items_capacity(10000)
    .weight_capacity(10000)
    .build()
    .unwrap(),
    UnitWeighter,
    DefaultHashBuilder::default(),
    DefaultLifecycle::default(),
);

The entry API allows you to perform atomic operations on a cache entry using a closure. This prevents race conditions between checking for a value and inserting/removing it.

Supported actions via EntryAction:

• Retain(value): Keep the existing entry.
• Remove: Evict the entry.
• Replace: (Implicitly handled via the guard in Vacant results).

EntryResult return types:

• Retained(v): The entry was present and retained.
• Vacant(guard): The entry was absent; use the provided guard to insert a new value.
• Removed(key, value): The entry was present but removed based on your logic.

use quick_cache::sync::{Cache, EntryAction, EntryResult};

fn main() {
    let cache: Cache<u64, u64> = Cache::new(100);

    // Insert-or-get: if absent, compute and insert; if present, return cached
    let result = cache.entry(&0, None, |_key, val| EntryAction::Retain(*val));
    let value = match result {
        EntryResult::Retained(v) => v,
        EntryResult::Vacant(guard) => {
            let v = 42; // expensive computation
            guard.insert(v).unwrap();
            v
        }
        _ => unreachable!(),
    };
    assert_eq!(value, 42);

    // Conditionally remove: evict entries below a threshold
    let result = cache.entry(&0, None, |_key, val| {
        if *val < 100 {
            EntryAction::<()>::Remove
        } else {
            EntryAction::Retain(())
        }
    });
    assert!(matches!(result, EntryResult::Removed(0, 42)));
}