sled Documentation

repository·main·Indexed 27 days ago

https://github.com/spacejam/sled

A lightweight, high-performance, thread-safe embedded database implemented in pure Rust. sled provides an API similar to BTreeMap with ACID transactions, zero-copy reads, and flash-optimized storage. It supports atomic compare-and-swap operations, asynchronous key prefix subscriptions, and configurable durability via manual or automatic flushing. The database allows for isolated keyspaces through multiple Trees within a single Db instance.

Tokens
2.5K
Snippets
5
Records
18
Agent score
94%

What's inside sled

  1. Understand the sled safety model and loss prevention

    main

    The sled embedded database is designed around a safety model aimed at preventing specific undesirable situations (losses). Developers using sled should be aware of these potential failure modes to design their applications with appropriate error handling and redundancy.

    Undesirable situations (Losses) to prevent:

    • Data loss: Permanent loss of stored information.
    • Inconsistent data access: Accessing data that is non-linearizable.
    • Process crash: Unexpected termination of the database process.
    • Resource exhaustion: Running out of storage or memory.
  2. Quickstart with sled

    main

    To use sled as an embedded database, open a tree and perform standard key-value operations like insert, get, remove, and range queries. It behaves similarly to a thread-safe BTreeMap<[u8], [u8]>.

    let tree = sled::open("/tmp/welcome-to-sled")?;
    
    // insert and get, similar to std's BTreeMap
    let old_value = tree.insert("key", "value")?;
    
    assert_eq!(
      tree.get(&"key")?,
      Some(sled::IVec::from("value")),
    );
    
    // range queries
    for kv_result in tree.range("key_1".."key_9") {}
    
    // deletion
    let old_value = tree.remove(&"key")?;
    
    // atomic compare and swap
    tree.compare_and_swap(
      "key",
      Some("current_value"),
      Some("new_value"),
    )?;
    
    // block until all operations are stable on disk
    tree.flush()?;
  3. Ensure data durability with Db::flush

    main
    Sled provides atomic, linearizable durability. All previous writes are guaranteed to be durable after a call to Db::flush returns Ok(()). A background flusher thread calls this periodically, but manual calls are required to ensure specific writes are persisted. Each call to Db::flush advances the current flush epoch by 1, ensuring that all writes in that epoch are committed atomically.
  4. Handle numeric keys with big-endian ordering

    main

    Because sled uses lexicographic ordering for its iterators and range queries, you must store numerical keys in big-endian form. Using little-endian (the default for many systems) will cause lexicographic ordering to diverge from numerical ordering once you exceed 256 items (1 byte).

    In Rust, use .to_be_bytes() to convert integral types to big-endian bytes.

  5. Configure durability and flushing

    main

    By default, sled automatically performs an fsync every 500ms. You can manage durability in two ways:

    1. Manual Flush: Call tree.flush() to block until operations are stable on disk, or tree.flush_async() to get a Future that you can await.
    2. Configuration: Adjust the automatic sync interval using the flush_every_ms configuration option.
  6. Configure sled cache size and eviction behavior

    main
    You can tune the in-memory cache behavior using the Config object. The overall cache size is controlled by cache_size. To manage how the cache handles scan-resistant workloads, you can adjust entry_cache_percent, which determines what percentage of the cache is reserved for leaves that are accessed at most once (the default is 20%).
  7. Open a sled database

    main

    Use sled::open to initialize a new database instance at the specified path. If the directory does not exist, it will be created. This returns a Db instance which can be used to manage multiple isolated trees.

    let db: sled::Db = sled::open("my_db").unwrap();
  8. Basic CRUD operations with Db and Tree

    main

    The Db and Tree types provide an API similar to BTreeMap<[u8], [u8]>. You can insert, get, remove, and range-scan keys and values. Note that Tree implements Deref<Target = Tree>, so a Db can often be used directly as a tree.

    let db: sled::Db = sled::open("my_db").unwrap();
    
    // insert and get
    db.insert(b"yo!", b"v1");
    assert_eq!(&db.get(b"yo!").unwrap().unwrap(), b"v1");
    
    // remove
    db.remove(b"yo!");
    assert!(db.get(b"yo!").unwrap().is_none());
    
    // range scan
    let scan_key: &[u8] = b"a non-present key before yo!";
    let mut iter = db.range(scan_key..);
    assert_eq!(&iter.next().unwrap().unwrap().0, b"yo!");
  9. Important usage warnings and limitations

    main

    Keep the following constraints in mind when using sled:

    • Single Instance: sled does not support multiple open instances of the same database. Keep the instance open for the duration of your process's lifespan.
    • Optimistic Transactions: Transactions are optimistic. Do not perform I/O or interact with external state inside a transaction closure unless the operation is idempotent.
    • Stability: sled is currently in beta and should be considered unstable. The on-disk format is subject to change before the 1.0.0 release, requiring manual migrations.
    • Use Cases:
      • For extreme reliability: use SQLite.
      • For storage price/performance: use RocksDB.
      • For multi-process workloads with rare writes: use LMDB.
      • sled is best for long-running, highly-concurrent workloads like stateful services.
  10. Use Db::apply_batch for atomic write batches

    main
    To ensure multiple writes are treated as a single atomic unit, use Db::apply_batch. After a crash recovery, all write batches processed via apply_batch will be either 100% visible or 0% visible. If the batch was followed by a successful Db::flush that returned Ok(()), the entire batch is guaranteed to be present.