DOCSEARCHbyAlgolia
Submit Docs

dynamodb-onetable

repository·main·Indexed 20 days ago

https://github.com/sensedeep/dynamodb-onetable

A DynamoDB access library for single-table designs that provides schema-driven models and type safety. It simplifies the creation and management of DynamoDB tables by allowing developers to define indexes, models, and attributes within a schema. The library supports CRUD operations, atomic additions, transactional updates, and integration with both AWS SDK V2 and V3.

Tokens
19.5K
Snippets
70
Records
92
Agent score
72%

What's inside dynamodb-onetable

  1. OneTable Schema Specification v1.0.0 Overview

    main

    The OneTable Schema is a JSON-based specification used to define application entities, their fields, and DynamoDB index structures for single-table designs. It allows the database to be self-describing, where schemas can be stored within the data table itself. This specification is used by the dynamodb-onetable library to map application models to physical DynamoDB attributes and keys.

    {
    "version": "0.1.0",
    "format": "onetable:1.0.0",
    "indexes": {
        "primary": { "hash": "PK", "sort": "SK" },
        "GSI1": { "hash": "GS1PK", "sort": "GS1SK" }
    },
    "params": {
        "typeField": "_type"
    },
    "models": {
        "Account": {
            "PK":          { "type": "string", "value": "account#${name}" },
            "SK":          { "type": "string", "value": "account#" },
            "name":        { "type": "string", "required": true }
        }
    }
}

  2. Use the SenseDeep Logging Access sample

    main

    The SenseDeep Logging Access sample demonstrates how to access and analyze logging data stored in a SenseDeep database table within your AWS account. It allows you to:

    • Retrieve and display a list of logs.
    • Select and display a sample of log events.
    • Retrieve a list of the most recent events for a specific log.

    The sample includes the SenseDeep OneTable schema definition in src/schema.js to facilitate these operations.

  3. Use Packed Attributes in OneTable

    main

    The Packed Attributes pattern involves packing multiple properties into a single DynamoDB attribute. This technique is particularly useful when you need to project a specific set of data into a Global Secondary Index (GSI) without creating multiple GSIs or increasing storage overhead unnecessarily. This sample demonstrates how to implement this pattern using a simple OneTable schema and the AWS SDK v3.

    For a detailed conceptual explanation of this pattern, refer to the original article: https://www.sensedeep.com/blog/posts/2021/attribute-packing.html

  4. OneTable Schema Specification Overview

    main
    The OneTable Schema is a JSON-based specification used to define application entities, their fields, and DynamoDB table structures (indexes) for single-table designs. It allows the database to be self-describing, as schemas can be stored directly in the data table. The specification is used by the dynamodb-onetable library to interpret and manage data items.

  5. Use the items property for schema visualization

    main

    The items property in a OneTable Schema is used to provide sample data.

    • Requirement: The items property MUST be an array of data items.
    • Constraint: Each data item MUST be a valid instance of an application entity as described in the models property of the schema.
    • Purpose: While optional, providing a small amount of data in items helps assist with visualizing the schema in various tools.

  6. Use Value Templates for Dynamic Attributes

    main

    The value property in a model entity allows you to construct attribute values at runtime using template strings. This is useful for composite keys (PK/SK).

    Syntax:

    • ${attribute}: Substitutes the value of the named attribute.
    • ${attribute:size}: Pads the value to a specific size.
    • ${attribute:size:pad}: Pads the value to a specific size using a custom character (defaults to 0). This is recommended for numbers to ensure correct lexicographical sorting in DynamoDB.

    Example: If an entity has a field id with value "${id:5:0}", and id is 42, the resulting value will be 00042.

  7. Define a OneTable Schema

    main

    Schemas define how items are stored, including index configurations and model definitions.

    Key schema components:

    • format: The OneTable version (e.g., onetable:1.1.0).
    • indexes: Defines primary, gs1 (GSI), and ls1 (LSI) indexes. Use follow: true for GSIs to enable specific behaviors.
    • models: Defines entities. Each model specifies its pk (partition key) and sk (sort key) using template strings (e.g., account:${id}).
    • attributes: Within models, you can define type, value (for keys), generate (e.g., ulid), required, default, enum, and validate (regex).
    • params: Global settings like isoDates and timestamps.
    const MySchema = {
    format: 'onetable:1.1.0',
    version: '0.0.1',
    indexes: {
        primary: {hash: 'pk', sort: 'sk'},
        gs1: {hash: 'gs1pk', sort: 'gs1sk', follow: true},
        ls1: {sort: 'id', type: 'local'},
    },
    models: {
        Account: {
            pk: {type: String, value: 'account:${id}'},
            sk: {type: String, value: 'account:'},
            id: {type: String, generate: 'ulid', validate: /^[0123456789ABCDEFGHJKMNPQRSTVWXYZ]{26}$/i},
            name: {type: String, required: true},
            status: {type: String, default: 'active'},
            zip: {type: String},
        },
        User: {
            pk: {type: String, value: 'account:${accountName}'},
            sk: {type: String, value: 'user:${email}', validate: EmailRegExp},
            id: {type: String, required: true},
            accountName: {type: String, required: true},
            email: {type: String, required: true},
            firstName: {type: String, required: true},
            lastName: {type: String, required: true},
            username: {type: String, required: true},
            role: {type: String, enum: ['user', 'admin'], required: true, default: 'user'},
            balance: {type: Number, default: 0},

            gs1pk: {type: String, value: 'user-email:${email}'},
            gs1sk: {type: String, value: 'user:'},
        },
    },
    params: {
        isoDates: true,
        timestamps: true,
    },
}

  8. Run the OneTable TypeScript sample

    main

    The OneTable TypeScript sample is a demonstration project that showcases core OneTable features including multi-tenant schemas, CRUD operations for various entities (Account, User, Product, Invoice), and common OneTable patterns using the AWS V3 SDK. It is configured to start its own DynamoDB instance on port 4567 and create its own DynamoDB table.

    make build
make run

  9. Set up the SenseDeep Logging Access sample

    main

    To use this sample, ensure you meet the following requirements:

    1. AWS Configuration: You must have SenseDeep activated and a cloud configured in your AWS account.
    2. Credentials: Set the AWS_PROFILE environment variable to point to your credentials in ~/.aws/credentials, or manually define your AWS credentials within src/index.js.
    3. Log Selection: You must modify src/index.js to specify the specific log group you wish to read events from.

  10. Explore OneTable samples

    main

    The dynamodb-onetable repository contains several working samples designed to help you learn OneTable and implement single-table designs for DynamoDB. Depending on your needs, you can explore different sample implementations:

    • Overview Sample: A quick tour through the core concepts of OneTable.
    • TypeScript Sample: A quick tour specifically demonstrating OneTable usage with TypeScript.
    • CRUD Sample: Demonstrates basic Create, Read, Update, and Delete operations.
    • Packed Attributes Sample: Shows how to implement and use packed attributes within your schema.
    • SenseDeep Sample: Demonstrates how to access SenseDeep log data.
    • Migration Controller: A separate project for managing OneTable Migrations.