FaunaDB

repository·main·Indexed 41 days ago

https://github.com/fauna/faunadb

An operational database combining document flexibility with native relational capabilities, strong consistency, and global distribution. It features the Fauna Query Language (FQL) for data manipulation, Fauna Schema Language (FSL) for schema-as-code, and ACID compliance via the Calvin protocol. The ecosystem includes an FQLX Language Server for LSP support and a TypeScript library for FQL autocompletions.

Tokens
9.2K
Snippets
32
Records
46
Agent score
49%

What's inside Fauna

  1. Overview of the FQL Language Library

    main

    The FQL Language Library provides tools to improve the developer experience when working with the Fauna Query Language (FQL). It includes:

    1. A TypeScript library designed for providing auto-completion for FQL.
    2. A standalone language server binary (LSP) for integration with IDEs and editors.
  2. Understand Fauna Cluster Topology (Compute vs Data+Log replicas)

    main

    Fauna clusters are organized by replica types. Understanding the difference is critical for scaling and availability:

    • Compute only replicas: These do not contain data. They act strictly as transaction coordinators to execute FQL queries.
    • Data+Log replicas: These contain a full copy of the cluster dataset, participate in the global log, and process scan-based background tasks. They can also execute FQL queries.

    When designing your topology, you can separate these concerns into different tiers.

  3. Security and Access Control in Fauna

    main

    Fauna provides two layers of access control:

    • Role-Based Access Control (RBAC): For coarse-grained permissions.
    • Attribute-Based Access Control (ABAC): For fine-grained, dynamic access rules.

    It includes built-in user authentication and supports integration with third-party identity providers.

  4. Change Data Capture (CDC) and Real-time Events

    main

    Fauna's CDC feature allows applications to react to data changes in real-time via two methods:

    • Real-time event streams: Atomic push updates.
    • Event feeds: On-demand batch pull updates.

    These are useful for maintaining live application states, building collaborative features, or mirroring changes to external systems.

  5. Transactions and Consistency in Fauna

    main

    Every query in Fauna is a transaction, ensuring ACID compliance. Fauna uses the Calvin protocol to provide:

    • Strict serializability for all read-write queries.
    • Serializable isolation for read-only queries.

    This consistency is maintained even across globally distributed region groups.

  6. Understand the Fauna Relational Model

    main

    Fauna combines document flexibility with relational capabilities:

    • Documents: Data is stored in JSON-like documents, removing the need for an ORM.
    • Collections: Documents are organized into collections (similar to relational tables).
    • Schema: Collections can specify document types to enforce structure, allowing for a transition from flexible to structured schemas.
    • Relationships: Fauna supports first-class relationships between documents in different collections, enabling complex modeling without data duplication.
  7. Use Fauna Query Language (FQL)

    main

    FQL is a TypeScript-inspired language for querying and manipulating data. It supports complex joins, data transformations, and optional static typing. A key feature is User-Defined Functions (UDFs), which allow you to encapsulate business logic within the database.

    Example of a relational query involving lookups and projections:

    // Gets the first customer with an email of "alice.appleseed@example.com".
    let customer = Customer.where(.email == "alice.appleseed@example.com")
                    .first()
    
    // Gets the first order for the customer, sorted by descending creation time.
    Order.where(.customer == customer)
      .order(desc(.createdAt)).
      first() {
        // Projecting fields resolves references, similar to a SQL join.
        customer {
          name,
          email
        },
        status,
        createdAt,
        items {
          product {
            name,
            price,
            stock,
            category {
              name
            }
          },
          quantity
        },
        total
      }
  8. Multi-tenancy and Environment Isolation

    main

    Fauna's database model supports isolation for multi-tenant applications and different deployment environments:

    • Multi-tenancy: Create isolated child databases for each tenant with separate access controls and schemas.
    • Environments: Create separate databases for staging and production to ensure changes in one do not affect others.
  9. Define schemas with Fauna Schema Language (FSL)

    main

    FSL allows you to manage database schema as code, enabling version control and CI/CD integration. You can define fields, computed fields, uniqueness constraints, and indexes.

    Example FSL schema for a Customer collection:

    collection Customer {
      name: String
      email: String
      address: {
        street: String
        city: String
        state: String
        postalCode: String
        country: String
      }
    
      compute cart: Order? = (customer => Order.byCustomerAndStatus(customer, 'cart').first())
    
      // Use a computed field to get the Set of Orders for a customer.
      compute orders: Set<Order> = ( customer => Order.byCustomer(customer))
    
      // Use a unique constraint to ensure no two customers have the same email.
      unique [.email]
    
      index byEmail {
        terms [.email]
      }
    }
  10. Create a new QA Test Generator

    main

    To add a new QA Test Generator for the database Concourse pipeline, you must create a directory in qa/DriverTestGenerators/ that follows these structural and naming rules:

    1. Naming: The directory name must exactly match a class definition in src/main/scala/qa/DriverTestGenerators.scala (case-sensitive).
    2. Language Subdirectories: The directory must contain at least one subdirectory for a supported driver language: js, python, go, java, or dotnet.
    3. Language Requirements: Each language subdirectory must include:
      • An application that uses the Fauna driver for that language to run queries against the database.
      • An init.sh script for setup (e.g., npm install). Note: This file must be executable (chmod +x).
      • A run.sh script that executes the application in its respective runtime (e.g., node index.js).
    4. Pipeline Integration: After creating the directory, you must add the new generator to the QA runs in ops/concourse/pipelines/core/pipeline.yml.
  11. Run FaunaDB development commands with sbt

    main

    Use the following sbt commands to manage the development lifecycle:

    • Compile all code and tests: sbt Test/compile
    • Run all tests: sbt test (Note: This may take a long time)
    • Run tests in a specific subproject (e.g., model): sbt model/test
    • Run specific test Specs using a glob: sbt model/testOnly *DatabaseSpec
    • Run a temporary database node: sbt runCore (Warning: This resets persisted state every time)
    # Compile all code and tests
    sbt Test/compile
    
    # Run all tests
    sbt test
    
    # Run tests in a specific subproject
    sbt model/test
    
    # Run test Specs matching a glob
    sbt model/testOnly *DatabaseSpec
    
    # Run a temporary database node
    sbt runCore
  12. Perform cluster backups and restores

    main

    Scheduled Backups

    Use a systemd service and timer to automate S3 uploads. The service requires the following environment variables:

    • S3_AWS_REGION
    • S3_AWS_PREFIX (e.g., s3://my/bucket/path)
    • SNAPSHOT_ROOT_DIR (e.g., /media/ephemeral/fauna/snapshots)

    Emergency Cluster Restoration

    To restore an entire cluster:

    1. Stop faunadb on all nodes.
    2. Clear all data directories (/media/ephemeral/fauna/data/*).
    3. Reinitialize the cluster starting from the first node using faunadb-admin init.
    4. Join all other nodes to the cluster.
    5. Update replication settings using faunadb-admin update-replica and faunadb-admin update-replication.
    6. Load the snapshot on the node containing the backup: faunadb-admin load-snapshot /path/to/snapshot.
    7. Update the storage version on all nodes: faunadb-admin update-storage-version.

    Manual Data Recovery

    If no backup is available, you can attempt to restore by combining deduplicated files from all nodes into a single directory structure:

    • HistoricalIndex-*
    • LookupStore-*
    • SortedIndex-*
    • Versions-*

    Then run faunadb-admin load-snapshot /path/to/combined-data.

    # Load a snapshot
    cd /opt/fauna/bin
    faunadb-admin load-snapshot /path/to/snapshot
    
    # After loading, update storage version
    faunadb-admin update-storage-version