LevelDB

LevelDB has specific concurrency constraints:

• Process Level: Only one process may open a database at a time. LevelDB uses OS locks to enforce this.
• Thread Level: A single leveldb::DB object is thread-safe and can be shared across multiple threads without external synchronization.
• Non-thread-safe objects: Objects like leveldb::Iterator and leveldb::WriteBatch are not thread-safe. If multiple threads must access the same instance of these objects, you must implement your own locking protocol.

To ensure multiple updates are applied atomically (all or nothing), use the leveldb::WriteBatch class. This prevents partial updates if a process crashes. WriteBatch also improves performance for bulk updates by grouping multiple mutations into a single write operation.

Note: When performing operations like moving a value from key1 to key2, call Delete(key1) before Put(key2, value) within the batch to avoid issues if key1 and key2 are identical.

#include "leveldb/write_batch.h"

std::string value;
leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value);
if (s.ok()) {
  leveldb::WriteBatch batch;
  batch.Delete(key1);
  batch.Put(key2, value);
  s = db->Write(leveldb::WriteOptions(), &batch);
}

Before using LevelDB, be aware of the following constraints:

• No Relational Model: It is not a SQL database. It does not support SQL queries or indexes.
• Single Process Access: Only one process (which may be multi-threaded) can access a specific database instance at a time.
• No Built-in Client-Server: LevelDB is a library, not a server. To provide client-server access, you must wrap the library in your own server implementation.

LevelDB uses leveldb::Slice for it->key() and it->value() to avoid expensive memory copies of large byte arrays. A Slice is a lightweight structure containing a pointer to an external byte array and a length.

Critical Safety Warning: A Slice does not own the data it points to. You must ensure the underlying memory (e.g., a std::string) remains valid for the entire lifetime of the Slice. If the source string goes out of scope, the Slice becomes a dangling pointer.

Conversions:

• std::string $\rightarrow$ Slice: Implicitly supported.
• Slice $\rightarrow$ std::string: Use .ToString().

// Conversion to Slice
leveldb::Slice s1 = "hello";
std::string str("world");
leveldb::Slice s2 = str;

// Conversion from Slice
std::string str_back = s1.ToString();

If a FilterPolicy is used when opening the database, a filter block is stored in the table. The metaindex maps the key filter.<N> (where <N> is the string from FilterPolicy::Name()) to the filter block's BlockHandle.

The filter block stores a sequence of filters. Each filter $i$ covers keys from data blocks whose file offsets fall within the range [ i*base ... (i+1)*base-1 ]. Currently, base is 2KB.

Filter Block Format:

1. A sequence of filters: [filter 0], [filter 1], ..., [filter N-1]
2. An offset array: [offset of filter 0], [offset of filter 1], ..., [offset of filter N-1] (each 4 bytes)
3. Metadata: [offset of beginning of offset array] (4 bytes) and lg(base) (1 byte).

Snapshots provide a consistent, read-only view of the entire database at a specific point in time. You can use a snapshot with leveldb::ReadOptions::snapshot to ensure that an iterator or a Get operation sees the state of the database as it existed when the snapshot was taken, even if other threads are performing writes.

Lifecycle: You must manually release snapshots using db->ReleaseSnapshot(snapshot_handle) when they are no longer needed to allow the engine to reclaim resources.

leveldb::ReadOptions options;
options.snapshot = db->GetSnapshot();

// ... apply updates to db ...

leveldb::Iterator* iter = db->NewIterator(options);
// ... read using iter to view the state when the snapshot was created ...

delete iter;
db->ReleaseSnapshot(options.snapshot);

LevelDB organizes sorted tables (*.ldb) into a hierarchy of levels to manage data migration and read performance:

• Level 0 (Young Level): Contains sorted tables generated directly from log files. Files in Level 0 may have overlapping key ranges.
• Levels 1 and above: Files in these levels have distinct, non-overlapping key ranges.
• Compaction Trigger: When the combined size of files in level $L$ exceeds $10^L$ MB (e.g., 10MB for level-1, 100MB for level-2), a compaction is triggered to merge files into level $L+1$.
• Data Migration: Compactions gradually migrate updates from the young level to the largest levels using bulk reads and writes to minimize expensive seeks.

A LevelDB database is represented as a set of files stored in a single directory. The key components are:

• Log files (*.log): Store a sequence of recent updates. When a log reaches its size threshold (default ~4MB), it is converted into a sorted table and a new log is created.
• Sorted tables (*.ldb): Store entries sorted by key (values or deletion markers). These are organized into levels.
• MANIFEST: A log file that tracks the set of sorted tables in each level, their key ranges, and other metadata. A new MANIFEST is created whenever the database is reopened.
• CURRENT: A text file containing the name of the latest MANIFEST file.
• Info logs (LOG, LOG.old): Files containing informational messages.
• Miscellaneous: LOCK and *.dbtmp files may also be present.