leveldown

LevelDB is a fast key-value storage library developed by Google. It provides an ordered mapping from string keys to string values using arbitrary byte arrays.

Core Features:

• Ordered Storage: Data is stored sorted by key. Users can provide a custom comparison function to override the default sort order.
• Atomic Batches: Multiple changes can be applied in a single atomic batch.
• Snapshots: Users can create transient snapshots to obtain a consistent view of the data.
• Iteration: Supports both forward and backward iteration over the data.
• Compression: Data is automatically compressed using the Snappy compression library.
• Extensible Environment: External activity (like file system operations) is handled through a virtual interface, allowing users to customize OS interactions.

Warning

leveldown is superseded by classic-level.

It is strongly recommended to use levelup in preference to leveldown unless you have specific, measurable performance reasons to do so. levelup is optimized for usability and safety. leveldown can still cause Node.js process crashes if operations are not performed correctly.

A LevelDB database is represented by a set of files stored in a single directory. The key components are:

• Log files (*.log): Store a sequence of recent updates. When a log file reaches its size threshold (default ~4MB), it is converted into a sorted table and a new log is started.
• Sorted tables (*.ldb): Store entries sorted by key (values or deletion markers). These are organized into levels.
• MANIFEST: A log file that lists the sorted tables in each level, their key ranges, and other metadata. A new MANIFEST is created whenever the database is reopened.
• CURRENT: A text file containing the name of the latest MANIFEST file.
• Info logs (LOG, LOG.old): Contain informational messages.
• Miscellaneous (LOCK, *.dbtmp): Files used for locking and temporary operations.

LevelDB has specific concurrency constraints:

• Process Level: A database can only be opened by one process at a time. LevelDB uses OS-level locks to enforce this.
• Thread Level: A single leveldb::DB object can be safely shared across multiple threads without external synchronization. Threads can call Get, Put, Delete, or create iterators concurrently.
• Object Level: Objects like leveldb::Iterator and leveldb::WriteBatch are not thread-safe. If multiple threads must access the same iterator or batch, you must provide your own external synchronization (e.g., mutexes).

leveldown supports LevelDB snapshots. A snapshot captures the state of the database at a specific point in time.

When using a snapshot (for example, by creating a createReadStream and a createWriteStream simultaneously), any data modified by the write stream will not affect the data emitted from the read stream. The snapshot allows you to iterate or read data without seeing subsequent writes. Any read operation not explicitly performed on a snapshot will implicitly use the latest state of the database.

Before using LevelDB, be aware of the following constraints:

• Not a SQL Database: It does not have a relational data model, does not support SQL queries, and has no support for indexes.
• Single Process Access: Only a single process (which may be multi-threaded) can access a particular database instance at a time.
• No Built-in Client-Server Support: LevelDB is a library, not a standalone server. If your application requires client-server architecture, you must wrap the library in your own server implementation.

To prevent partial updates (e.g., if a process crashes between a Put and a Delete), use the leveldb::WriteBatch class. A batch allows you to group multiple edits that are applied atomically. This also improves performance for bulk updates.

Note: When performing operations like moving a value from key1 to key2, call Delete(key1) before Put(key2, value) within the batch to avoid data loss if the keys are identical.

#include "leveldb/write_batch.h"
... 
std::string value;
leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value);
if (s.ok()) {
  leveldb::WriteBatch batch;
  batch.Delete(key1);
  batch.Put(key2, value);
  s = db->Write(leveldb::WriteOptions(), &batch);
}

When a database is opened, LevelDB follows these steps to ensure consistency:

1. Read the CURRENT file to find the name of the latest committed MANIFEST.
2. Read that MANIFEST file.
3. Clean up stale files (garbage collection).
4. Convert the existing log chunk into a new Level 0 sorted table.
5. Start directing new writes to a new log file using the recovered sequence number.

A leveldb::Slice is a lightweight structure containing a pointer and a length. It is used for keys and values to avoid expensive memory copies of large byte arrays.

Key Behaviors:

• Conversion: You can convert std::string to Slice and Slice to std::string via .ToString().
• Safety Warning: A Slice does not own the underlying data. You must ensure the source buffer (e.g., a std::string) remains in scope and is not destroyed while the Slice is being used. Using a Slice that points to a local variable that goes out of scope is a common source of bugs.

// String to Slice
leveldb::Slice s1 = "hello";
std::string str("world");
leveldb::Slice s2 = str;

// Slice to String
std::string str_from_s1 = s1.ToString();