go-testfixtures/testfixtures

repository·master·Indexed 23 days ago

https://github.com/go-testfixtures/testfixtures

A Go package for managing database state in functional tests using YAML fixtures. It provides tools to clean and reload sample data, supporting multiple dialects including PostgreSQL, MySQL, MariaDB, SQLite, MS SQL Server, and Google Spanner. Features include a CLI for loading and dumping fixtures, YAML templating for dynamic data, and a dumper to bootstrap fixtures from existing databases.

Tokens
5.6K
Snippets
12
Records
32
Agent score
79%

What's inside testfixtures

  1. Create a fresh logical database from a template approach

    master

    This approach involves creating a database template once, and then having each test create a new logical database cloned from that template. This can be done using helpers like github.com/peterldowns/pgtestdb or via database-specific SQL commands.

    Pros:

    • High isolation: the physical instance is shared, but logical databases are separate.
    • Fast: assuming the template cloning process is efficient.
    • Good for parallel execution.

    Cons:

    • Variable speed: cloning speed depends on the complexity of the template.
    • Setup overhead: requires preparing a template, creating test databases from it, and cleaning them up.
    • Engine specific: requires a database-specific library or manual implementation of cloning logic.
    -- create and prepare a template database
    CREATE DATABASE dbname TEMPLATE template;
    -- use your dbname database during test
  2. Run each test in a transaction approach

    master

    In this approach, you use a single database, but each test starts a new transaction that is ROLLBACKed once the test completes. This prevents tests from interfering with each other's data.

    You can manage transactions manually or use a library like github.com/DATA-DOG/go-txdb, which wraps the *sql.DB interface with a transaction manager.

    Pros:

    • Extremely fast: likely the fastest method when applied correctly.
    • Good for parallel execution.
    • Easy to set up.

    Cons:

    • Engine limitations: go-txdb currently only supports postgres and mysql.
    • Isolation risks: Transactions may not isolate tests during DDL (Data Definition Language) operations, which testfixtures uses heavily.
    • Performance bottlenecks: DDL operations can slow down or lock the database, potentially making the suite as slow as sequential testing.
    • Complexity: may not work for nested transactions (transaction in transaction).
  3. Separate database per test approach

    master

    This approach creates a completely disposable database for every individual test. This is typically achieved using container orchestration tools like testcontainers-go or dockertest to spin up a fresh Docker container for each test.

    Pros:

    • Perfect isolation between tests.
    • Excellent for parallel execution.
    • Works with any database engine.

    Cons:

    • Slow setup: booting a database container can take anywhere from 1 second (e.g., postgres) to over 10 seconds.
    • Infrastructure requirement: requires a docker runtime or similar container orchestration environment.
  4. Format data in YAML fixtures

    master

    testfixtures supports several special data formats in your YAML files:

    • JSON/Objects: YAML objects or arrays are converted to JSON and stored in native JSON types (like JSONB in PostgreSQL) or TEXT/VARCHAR columns.
    • Binary Data: Represented as hexadecimal strings starting with 0x (e.g., 0x1234567890abcdef).
    • Date/Time: Strings matching date/time formats are automatically converted to time.Time.
    • Raw SQL: Prefix a value with RAW= to prevent conversion and execute the value as raw SQL (useful for functions like NOW() or uuid_generate_v4()).
    • Preventing Time Conversion: Use the RAW= prefix to keep a date/time string as a literal string instead of converting it to time.Time.
  5. Sequential single-threaded testing approach

    master

    This approach uses a single shared database connection for all tests. You store the database connection and the fixtures object in a global variable so each test can access them.

    Pros:

    • Simple setup: connect once and use everywhere.
    • Performance: reusing the same fixtures object instance speeds up subsequent fixtures.Load() calls.
    • Compatibility: works with any database engine.

    Cons:

    • No parallelization: this causes significant performance issues as the test suite grows.
    • Global dependency: relies on global state.

    Requirements for correctness:

    • Do not use t.Parallel() in your database tests.
    • If you have database tests in multiple packages, use go test -p 1 ./... to ensure only one package is tested at a time.
  6. Generate fixtures from an existing database

    master

    You can bootstrap test scenarios by dumping an existing database into YAML files using testfixtures.NewDumper. This is intended for small sample databases and may fail on large production databases.

    dumper, err := testfixtures.NewDumper(
            testfixtures.DumpDatabase(db),
            testfixtures.DumpDialect("postgres"),
            testfixtures.DumpDirectory("tmp/fixtures"),
            testfixtures.DumpTables(
              "posts",
              "comments",
              "tags",
            ),
    )
    if err != nil {
            ...
    }
    if err := dumper.Dump(); err != nil {
            ...
    }
  7. How to use testfixtures for database testing

    master

    testfixtures allows you to run functional tests against a real database by loading sample data from YAML files. Before each test, it cleans the database and loads the specified fixtures.

    1. Organize Fixture Files

    By default, you can create a directory where each .yml file corresponds to a single table. The filename must match the table name (e.g., posts.yml for the posts table).

    2. Initialize the Loader

    Use testfixtures.New() to create a *testfixtures.Loader. You must provide a database connection and a dialect.

    3. Load Fixtures

    Call fixtures.Load() before your tests to wipe the database and load the data.

    Warning: This package wipes database data. Ensure you are running it against a dedicated test database.

    package myapp
    
    import (
            "database/sql"
            "os"
            "testing"
    
            _ "github.com/lib/pq"
            "github.com/go-testfixtures/testfixtures/v3"
    )
    
    var (
            db *sql.DB
            fixtures *testfixtures.Loader
    )
    
    func TestMain(m *testing.M) {
            var err error
    
            // Open connection to the test database.
            db, err = sql.Open("postgres", "dbname=myapp_test")
            if err != nil {
                    // handle error
            }
    
            fixtures, err = testfixtures.New(
                    testfixtures.Database(db),
                    testfixtures.Dialect("postgres"),
                    testfixtures.Directory("testdata/fixtures"),
            )
            if err != nil {
                    // handle error
            }
    
            os.Exit(m.Run())
    }
    
    func prepareTestDatabase() {
            if err := fixtures.Load(); err != nil {
                    // handle error
            }
    }
    
    func TestX(t *testing.T) {
            prepareTestDatabase()
            // Your test here ...
    }
  8. Enable YAML templating for dynamic data

    master
    Templating is disabled by default. To use it, call testfixtures.Template(). You can also provide custom functions, delimiters, options, and data. This allows you to use logic like {{range}} or custom functions like {{sha256 ...}} inside your YAML files.
  9. Use the testfixtures CLI to load or dump fixtures

    master

    The testfixtures CLI tool allows you to either load YAML fixtures into a database or dump the current state of a database into YAML files.

    Loading Fixtures

    To load fixtures, you must provide a database dialect and a connection string. You can specify the source of your fixtures using --dir (a directory), --files (specific YAML files), or --paths (a list of directories or files).

    Dumping Fixtures

    To dump the current database state into YAML files, use the --dump flag and specify a target directory with --dir.

    Requirements

    • You must provide either --dialect (-d) or --conn (-c).
    • If using --dump, you must provide --dir (-D).
    • If not dumping, you must provide --dir, --files, or --paths.
  10. Configure Microsoft SQL Server dialect

    master

    Supports SQL Server >= 2008 and handles IDENTITY column insertions. Ensure the user has ALTER TABLE permissions. Tested with mssql and sqlserver drivers from github.com/denisenkom/go-mssqldb.

    testfixtures.New(
            ...
            testfixtures.Dialect("sqlserver"),
    )