repository·main·Indexed 25 days ago
https://github.com/moka-rs/mokaA high-performance, concurrent caching library for Rust inspired by Java's Caffeine. Moka provides both synchronous (moka::sync::Cache) and asynchronous (moka::future::Cache) implementations featuring size-aware eviction, various expiration policies (TTL, TTI, and per-entry), and an atomic Entry API. It supports most 64-bit and 32-bit platforms where the Rust std library is available, though it does not support WebAssembly, WASI, or nostd environments.
Each example in the repository is a standalone binary. To run an example, use the following command format, ensuring you include the necessary features (sync and future):
$ cargo run --example <example_name> -F sync,futureNote the naming convention for examples:
_async suffix: Uses moka::future::Cache (a Future-aware, concurrent cache)._sync suffix: Uses moka::sync::Cache (a multi-thread safe, concurrent cache).For both sync and future caches, the get method returns Option<V> (a clone of the value) rather than Option<&V>. This is because the cache allows concurrent updates, and a reference cannot be guaranteed to outlive the potential replacement of the value by another thread.
To avoid the performance penalty of cloning large values, wrap your values in std::sync::Arc before inserting them. Cloning an Arc is a cheap, thread-safe operation.
v0.12.0 introduced major breaking changes to the API and internal behavior. Key changes include:
sync caches are no longer enabled by default: You must enable the sync crate feature to use sync::Cache or sync::SegmentedCache.future::Cache, sync::Cache, and sync::SegmentedCache) no longer spawn background threads. This means maintenance tasks (like removing expired entries) are now executed in the foreground during certain cache operations or must be triggered manually.notification::DeliveryMode enum was removed. All caches now behave as if Immediate delivery mode is specified for eviction listeners.future module were converted to async methods and now require .await.Synchronous caches are defined in the moka::sync module. They are thread-safe and can be shared across threads by cloning the cache instance (which is a cheap operation).
Common operations include:
insert(key, value): Manually add an entry.get(&key): Retrieve a value (returns Option<V>, providing a clone of the stored value).invalidate(&key): Manually remove an entry.get_with(key, loader): Atomically initialize and insert a value if the key is not present.try_get_with(key, loader): Similar to get_with but returns a Result.Note: v0.12.0 introduced major breaking changes. Refer to MIGRATION-GUIDE.md if upgrading from older versions.
use moka::sync::Cache;
use std::thread;
fn value(n: usize) -> String {
format!("value {n}")
}
fn main() {
const NUM_THREADS: usize = 16;
const NUM_KEYS_PER_THREAD: usize = 64;
// Create a cache that can store up to 10,000 entries.
let cache = Cache::new(10_000);
// Spawn threads and read and update the cache simultaneously.
let threads: Vec<_> = (0..NUM_THREADS)
.map(|i| {
// To share the same cache across the threads, clone it.
let my_cache = cache.clone();
let start = i * NUM_KEYS_PER_THREAD;
let end = (i + 1) * NUM_KEYS_PER_THREAD;
thread::spawn(move || {
for key in start..end {
my_cache.insert(key, value(key));
assert_eq!(my_cache.get(&key), Some(value(key)));
}
for key in (start..end).step_by(4) {
my_cache.invalidate(&key);
}
})
})
.collect();
threads.into_iter().for_each(|t| t.join().expect("Failed"));
for key in 0..(NUM_THREADS * NUM_KEYS_PER_THREAD) {
if key % 4 == 0 {
assert_eq!(cache.get(&key), None);
} else {
assert_eq!(cache.get(&key), Some(value(key)));
}
}
}The basic Cache API allows you to share a cache between async tasks or OS threads.
Important: Do not wrap a Cache with Arc<Mutex<_>>. Instead, simply clone the Cache instance to share it across threads or tasks.
Core methods include:
insert: Add an entry to the cache.get: Retrieve an entry.invalidate: Remove an entry.Since the .blocking() method was removed from future::Cache, you must use your async runtime's blocking implementation to call async cache methods from a synchronous context.
Obtain a handle to the current Tokio runtime using tokio::runtime::Handle::current() and use rt.block_on(...).
use std::sync::Arc;
#[tokio::main]
async fn main() {
// Create a future cache.
let cache = Arc::new(moka::future::Cache::new(100));
// In async context, you can obtain a handle to the current Tokio runtime.
let rt = tokio::runtime::Handle::current();
// Spawn an OS thread. Pass the handle and cache.
let thread = {
let cache = Arc::clone(&cache);
std::thread::spawn(move || {
// Call async function using block_on method of Tokio runtime.
rt.block_on(cache.insert(0, 'a'));
})
};
// Wait for the threads to complete.
thread.join().unwrap();
// Check the result.
assert_eq!(cache.get(&0).await, Some('a'));
}Use async_std::task::block_on to call async cache functions.
use std::sync::Arc;
#[tokio::main]
async fn main() {
// Create a future cache.
let cache = Arc::new(moka::future::Cache::new(100));
// In async context, you can obtain a handle to the current Tokio runtime.
let rt = tokio::runtime::Handle::current();
// Spawn an OS thread. Pass the handle and cache.
let thread = {
let cache = Arc::clone(&cache);
std::thread::spawn(move || {
// Call async function using block_on method of Tokio runtime.
rt.block_on(cache.insert(0, 'a'));
})
};
// Wait for the threads to complete.
thread.join().unwrap();
// Check the result.
assert_eq!(cache.get(&0).await, Some('a'));
}If your cache entries have varying memory footprints, you can use a weigher to define their relative size. The cache will evict entries when the total weighted size exceeds the configured max_capacity.
weigher is a closure that takes (&K, &V) and returns a u32 representing the weight.max_capacity defines the maximum total weight allowed.use moka::sync::Cache;
let cache = Cache::builder()
// Define weight based on the length of the String value
.weigher(|_key, value: &String| -> u32 {
value.len().try_into().unwrap_or(u32::MAX)
})
// Set max capacity to 32MiB
.max_capacity(32 * 1024 * 1024)
.build();
cache.insert(0, "zero".to_string());To use Moka, add it to your dependencies using cargo add. You must specify the feature corresponding to your required cache type:
tokio or async-std).# To use the synchronous cache:
cargo add moka --features sync
# To use the asynchronous cache:
cargo add moka --features futureIn v0.12, synchronous caches are disabled by default. To use sync::Cache or sync::SegmentedCache, you must enable the sync feature in your Cargo.toml.
Additionally, because background threads are removed:
thread_pool_enabled method of sync::CacheBuilder has been removed (the thread pool is always disabled).sync method of the sync::ConcurrentCacheExt trait has been moved to the sync::Cache and sync::SegmentedCache types and renamed to run_pending_tasks.Moka allows you to react to cache events like expiration and eviction.
Expiration Policies:
time_to_live policy.moka::Expiry trait (e.g., adding jitter to expiry durations).Eviction Listeners:
run_pending_tasks method to ensure expired entries are actually evicted from the cache in a timely manner.Use Cases:
BTreeMap) using an eviction listener.mpsc).The asynchronous (futures-aware) cache is located in the moka::future module and is compatible with runtimes like Tokio, async-std, or actix-rt.
To use it, you must enable the future crate feature in your Cargo.toml.
Usage Patterns:
insert or invalidate and .await them..blocking() method to access blocking versions of insert or invalidate.get_with or try_get_with to initialize and insert a value only if the key is not present.Moka uses entry replacement algorithms to manage capacity. You can choose between two main policies:
Both policies support bounding the cache by either the maximum number of entries or the total weighted size of entries.