CubDB Documentation

CubDB is an embedded, persistent database designed for Elixir applications. It functions like an Elixir collection (e.g., a Map or List) but is stored on disk and survives application restarts.

Key benefits include:

• Native Elixir Experience: Simple, idiomatic API that integrates with Elixir supervision trees.
• No Native Dependencies: Runs wherever Elixir runs without the need for cross-compiling native code (unlike SQLite or LevelDB), making it ideal for embedded systems like Nerves.
• Robustness: Designed to be efficient and robust against power loss, ensuring data is not corrupted and atomicity is maintained.

Typical Use Cases:

• Persisting configuration and user preferences.
• Data logging and storing metrics or time-series data.
• Serving as a local database for application data in single-instance applications.

When choosing a storage mechanism, consider how CubDB compares to these common alternatives:

CubDB uses an append-only B-tree data structure. Instead of modifying data in-place, every change is appended to the end of the file. This ensures high write performance and protects against data corruption during power failures.

The Trade-off: Because changes are appended, the data file grows over time even if you update or delete values (stale data remains in the file).

Compaction is the process of cleaning up this stale data to reclaim space. During compaction:

1. CubDB creates a new file.
2. It transfers only the current (reachable) entries to the new file.
3. Once complete, it switches to the new file and removes the old one.

Compaction runs in the background and does not block read or write operations. If interrupted by a crash, no data is lost; the system simply resumes or restarts the process on the next attempt.

When you need to perform multiple reads or selects while ensuring isolation from concurrent writes (without blocking those writes), use CubDB.with_snapshot/2. Inside the callback, use the CubDB.Snapshot module to access data. This implements Multi-Version Concurrency Control (MVCC).

# Ensuring consistency by getting both entries from the same snapshot
{x, y} = CubDB.with_snapshot(db, fn snap ->
  x = CubDB.Snapshot.get(snap, :x)
  y = CubDB.Snapshot.get(snap, x)

  {x, y}
end)

To use CubDB in your Elixir project, add :cubdb to your deps list in mix.exs.

def deps do
  [
    {:cubdb, "~> 2.0.2"}
  ]
end

You can create a snapshot of your database state and restore it later using the following workflow:

1. Create a backup: Use CubDB.back_up/2 to copy the current database state to a target directory.
2. Restore/Open backup: Start a new CubDB process by pointing CubDB.start_link/1 to the directory where the backup was created.

This is useful for migrations, snapshots, or creating isolated environments for testing.

# Backup the current state of the database
:ok = CubDB.back_up(db, "some/target/path")

# Open the backup as another CubDB process
{:ok, copy} = CubDB.start_link(data_dir: "some/target/path")

Initialize CubDB by calling CubDB.start_link/1 and providing a data_dir. If the directory does not exist, it will be created.

Warning: Avoid starting multiple CubDB processes on the same data directory. Only one CubDB process should use a specific data directory at any time.

{:ok, db} = CubDB.start_link(data_dir: "my/data/directory")

In v2, the select function no longer accepts :pipe, :reduce, or :timeout options. Instead, select returns a lazy stream. To perform transformations or reductions, use the Stream and Enum modules on the returned value.

# This v1 code:
{:ok, product} =
  CubDB.select(db, [
    min_key: :foo,
    max_key: :bar,
    pipe: [
      map: fn {_, val} -> val end,
      filter: fn val -> val > 0 end
    ],
    reduce: fn val, acc -> val * acc end
  ])

# Can be rewritten to this code in v2:
product =
  CubDB.select(db, min_key: :foo, max_key: :bar)
  |> Stream.map(fn {_, val} -> val end)
  |> Stream.filter(fn val -> val > 0 end)
  |> Enum.reduce(fn val, acc -> val * acc end)