pgtestdb

repository·main·Indexed 19 days ago

https://github.com/peterldowns/pgtestdb

A Go library for high-performance, isolated database testing that leverages PostgreSQL template databases to provide fresh, fully migrated database instances in milliseconds. It includes a variety of migrator packages for integration with Atlas (Versioned and Declarative workflows), uptrace/bun, amacneil/dbmate, golang-migrate/migrate, and pressly/goose.

Tokens
18.6K
Snippets
73
Records
97
Agent score
60%

What's inside pgtestdb

  1. Use sqlmigrator with rubenv/sql-migrate

    main

    The sqlmigrator package provides migrators for projects using rubenv/sql-migrate.

    Instead of relying on a global migration instance, sqlmigrator requires you to pass in a *migrate.MigrationSet. This approach ensures that migrations are parallel and concurrency safe.

    You can configure standard sql-migrate settings such as the migrations directory, the migration table name, and the filesystem being used.

  2. Implement a custom Migrator

    main

    If you need to support a migration framework not listed above, you can implement your own Migrator.

    When implementing a custom migrator, note that most require file or directory hashing to implement the Hash() method. It is recommended to use the helpers provided in the common subpackage to handle these operations.

  3. How pgtestdb works

    main

    pgtestdb provides isolated, fully migrated Postgres databases for each test using PostgreSQL template databases.

    Lifecycle:

    1. Template Creation: When pgtestdb.New is called, it checks if a template database exists. If not, it creates one and runs your migrations on it.
    2. Test Database Provisioning: It creates a new database from that template. This is extremely fast (~10-20ms).
    3. Cleanup: When a test succeeds, the database is automatically deleted. If a test fails, the database is left alive, and the connection string is printed to the logs so you can inspect it with psql.

    Key Features:

    • Concurrency Safe: Since every test gets its own database, you can run tests in parallel using t.Parallel().
    • Efficient Migrations: Migrations are hashed and run only once per unique set of migrations, even across different test runs or packages.
    • No Mocking: You interact with a real Postgres instance.
  4. Use goosemigrator with SQL migrations

    main

    The goosemigrator package allows you to run pressly/goose SQL migrations against a pgtestdb instance.

    Supported Features:

    • SQL Migrations: Only SQL-based migrations are supported.
    • Storage: Migrations can be read from a directory on disk or from an embed.FS (embedded filesystem).

    Limitations:

    • Golang-defined migrations: Migrations defined in Go code are not supported.
  5. Define a database helper for tests

    main

    To avoid repeating configuration in every test, define a helper function that returns a *sql.DB. This helper should call t.Helper() and use pgtestdb.New with your standard configuration.

    func NewDB(t *testing.T) *sql.DB {
      t.Helper()
      conf := pgtestdb.Config{
        DriverName: "pgx",
        User:       "postgres",
        Password:   "password",
        Host:       "localhost",
        Port:       "5433",
        Options:    "sslmode=disable",
      }
      var migrator pgtestdb.Migrator = pgtestdb.NoopMigrator{}
      return pgtestdb.New(t, conf, migrator)
    }
    
    func TestAQuery(t *testing.T) {
      t.Parallel()
      db := NewDB(t)
    
      var result string
      err := db.QueryRow("SELECT 'hello world'").Scan(&result)
      check.Nil(t, err)
      check.Equal(t, "hello world", result)
    }
  6. Configure goosemigrator with disk or embedded filesystem

    main

    You can initialize a migrator using goosemigrator.New(). You can configure the migrations directory, the migration table name, and the filesystem used via functional options.

    Migrations from a disk directory

    Pass the directory name as the first argument to goosemigrator.New().

    func TestGooseMigratorFromDisk(t *testing.T) {
      m := goosemigrator.New("migrations")
      db := pgtestdb.New(t, pgtestdb.Config{
        DriverName: "pgx",
        Host:       "localhost",
        User:       "postgres",
        Password:   "password",
        Port:       "5433",
        Options:    "sslmode=disable",
      }, m)
      assert.NotEqual(t, nil, db)
    }

    Migrations from an embedded filesystem

    Use goosemigrator.WithFS(fs) to provide an embed.FS and goosemigrator.WithTableName(name) to specify a custom migration table name.

    //go:embed migrations/*.sql
    var exampleFS embed.FS
    
    func TestGooseMigratorFromFS(t *testing.T) {
      gm := goosemigrator.New(
        "migrations",
        goosemigrator.WithFS(exampleFS),
        goosemigrator.WithTableName("goose_example_migrations"),
      )
      db := pgtestdb.New(t, pgtestdb.Config{
        DriverName: "pgx",
        Host:       "localhost",
        User:       "postgres",
        Password:   "password",
        Port:       "5433",
        Options:    "sslmode=disable",
      }, gm)
      assert.NotEqual(t, nil, db)
    }
  7. Use golangmigrator with pgtestdb

    main

    The golangmigrator provides a migrator compatible with projects using golang-migrate/migrate. It is designed to be passed into pgtestdb.New to automatically handle database migrations during test setup.

    Important Constraint: Because the Hash() method must calculate a unique hash based on migration contents, this implementation only supports reading migration files from the local disk or an embed.FS (embedded filesystem).

    //go:embed migrations/*.sql
    var exampleFS embed.FS
    
    func TestMigrateFromEmbeddedFS(t *testing.T) { 
      // Initialize migrator with embedded filesystem
      gm := golangmigrator.New(
        "migrations",
        golangmigrator.WithFS(exampleFS),
      )
    
      // Pass the migrator to pgtestdb
      db := pgtestdb.New(t, pgtestdb.Config{
        Host:     "localhost",
        User:     "postgres",
        Password: "password",
        Port:     "5433",
        Options:  "sslmode=disable",
      }, gm)
      assert.NotEqual(t, nil, db)
    }
  8. Use TernMigrator with pgtestdb

    main

    The ternmigrator package provides a migrator compatible with tern migrations. You can use it with pgtestdb.New by passing a ternmigrator instance as the third argument.

    By default, ternmigrator.New(dir) looks for migrations in the specified directory on the local filesystem. You can customize the migrator using functional options like WithFS to use an embed.FS or WithTableName to specify a custom migration tracking table.

    // Example: Using TernMigrator with an embedded filesystem
    //go:embed migrations/*.sql
    var exampleFS embed.FS
    
    func TestTernMigratorFromFS(t *testing.T) {
    	ctx := context.Background()
    
    	// Initialize migrator with directory and embedded FS
    	m := ternmigrator.New("migrations", ternmigrator.WithFS(exampleFS))
    
    	// Pass the migrator to pgtestdb.New
    	db := pgtestdb.New(t, pgtestdb.Config{
    		DriverName: "pgx",
    		Host:       "localhost",
    		User:       "postgres",
    		Password:   "password",
    		Port:       "5433",
    		Options:    "sslmode=disable",
    	}, m)
    
    	// The database is now automatically migrated using Tern
    }