DOCSEARCHbyAlgolia
Submit Docs

WedgeUp Documentation

website·Indexed 31 days ago

https://docs.wedgeup.com/

Documentation for a data generation tool that uses YAML-based schema and plan files to create meaningful datasets. It supports various dialects including PostgreSQL, MySQL, and Salesforce, and provides features for data providers, constraints, value distribution, and output control to ensure generated data meets specific requirements.

Tokens
12K
Snippets
68
Records
90
Agent score
50%

What's inside WedgeUp

  1. Overview of WedgeUp Data Providers

    Data providers are constraints applied to text properties to instruct the generator to create meaningful data. They are defined using the category type.

    Common Category Properties:

    • name (required): The name of the data provider.
    • qualifier (optional): Used to extract specific parts of the data (e.g., first name vs last name).
    • locale (optional): Limits generation to a specific IETF BCP 47 language tag (e.g., en-US, ja-JP).
    • compoundId (optional): Differentiates the same provider within one entity. Properties with the same category name and compoundId will share the same generated value. Defaults to 0.

    Important Constraint: Data providers can only be applied to properties of type text. They are ignored for other data types.

  2. Overview of WedgeUp data constraints

    Constraints are rules used to ensure generated data meets target data model requirements. They can be defined in either the schema file (for system-level constraints) or the plan file (for purpose-specific constraints). Constraints applied to an incompatible type are ignored.

  3. Use string alternation constraints to manipulate string values

    The alternation constraint allows you to generate new string values based on existing property values. While it can be used in schema files, it is recommended for use in plan configuration files. An alternation constraint requires three components: type (must be "alternation"), properties (the list of properties used to generate the value), and a formula defining the transformation.

  4. Define property specifications for entity data

    Property Specification is an optional feature used to establish constraints, distributions, and default values for individual properties. It consists of the following properties:

    • name: The target property name.
    • values: A list of property value distributions (e.g., specifying that 30% of values should be null).
    • defaultValue: A string representation of a default value for the target property.
    • referenceStrategy: Defines how referenced values are applied if the target property is a reference property.
    • constraints: A list of additional constraints to be applied to the property.

  5. Configure entities for data generation

    An entity is the fundamental unit of data generation, representing a distinct object in the data model. Each entity's data is stored in its own output file. An entity requires:

    • name: A unique identifier for the entity.
    • properties: A set of at least one property definition.

  6. Understand the WedgeUp data generation input files

    Data generation orders utilize up to four input files in YAML or CSV format:

    1. Schema File (Required): Defines the structure of the data to be generated.
    2. Plan File (Required): Defines target data details, such as record counts and generation strategies.
    3. Output Control File (Optional): Defines modifications for data presentation (e.g., name replacements).
    4. Data CSV File (Optional): Provides external customer data to be injected into the generation process.

  7. Generate an extra JSON output file

    An extra.json file can be generated by setting type: json in the extra configuration.

    • Schema Version 1 (Legacy): Uses label with dot notation (e.g., parent.child) to define the JSON hierarchy.
    • Schema Version 2: Set schemaVersion: 2 to use a more complex node-based structure involving Value Nodes and Entity Nodes, allowing for maps, arrays, and aggregations.

  8. Generate Salesforce account and contact data for a healthcare system

    This example demonstrates how to generate data for a healthcare system consisting of 5 business units and a total of 1,000 contacts. To implement this, use the provided schema, plan, and output control files. The generation process produces a data.zip containing CSV files that must be uploaded to Salesforce in a specific sequence to maintain referential integrity: first account_0.csv, then contact_1.csv.

  9. Rename entities and properties in output files

    The output control file allows you to define aliases for entities and their properties, which are applied after data generation. This is useful for mapping internal data model names to specific external requirements or handling special characters.
    name: "name_replacement"
entities:
  - name: "businessAccount"
    alias: "account"
    properties:
      - name: "extId"
        alias: "external_id"

  10. Define data generation strategy using the Plan File

    The Plan File instructs the generator on how to use a specific schema. It requires:

    • name: The name of the data plan.
    • schema: Must match the name defined in the corresponding Schema File.
    • data: A list of entity definitions, each requiring a count (the number of records to generate).
    • dialect (Optional): The target dialect. Supported values are general (default), mysql, and salesforce.
    name: "quickstart"
dialect: "general"
schema: "quickstart"
data:
  - entity: "users"
    count: 10

  11. Simulate customer email update data flows

    To test or validate data flows, you can combine user data input and extra data files to create interconnected data generation processes. In a customer email update scenario, the application typically requires a PostgreSQL database for customer records and two REST endpoints: one for authentication (POST /customers/auth) and one for updating the email (PUT /customers/password).
    // Auth Request
{
  "user": "username - text",
  "password": "password - text"
}

// Email Update Request
{
  "id": "customer id in database - integer",
  "email": "current email - text",
  "newEmail": "new email - text"
}

  12. Map Salesforce fields during Order data import

    When importing the generated CSV files into Salesforce (e.g., via dataloader.io), use the following lookup mappings to maintain entity relationships:

    • Contact records: Map Account ID using Lookup via: Account Number.
    • Contract records: Map Account ID using Lookup via: Account Number.
    • Order records:
      • Map Bill To Contract ID using Lookup via: Contract ExtId.
      • Map Account ID using Lookup via: Account Number.
      • Map Contact ID using Lookup via: Contact ExtId.