Go Transaction Manager

The trm/internal/codegen/redis directory contains instructions for a code generation process that creates Go wrappers for Redis commands. This process involves filtering existing commands and applying regex-based transformations to map them to a WritePipeliner or similar structure.

Codegen Workflow

1. Identify Read-Only Commands: Use a Go script to query the Redis client for commands and filter those where ReadOnly is true.
2. Filter Commands: Remove lines from the source that do not match the desired command pattern (e.g., using the regex ^(?!(?:del|set...)\().*).
3. Apply Command Transformation: Use a regex replacement to wrap the commands into a function signature that accepts a *WritePipeliner and returns a Redis-prefixed call.
4. Transform Arguments: Apply regex patterns to adjust how arguments are passed to the generated functions (e.g., adding *redis. prefixes to specific types).

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/go-redis/redis/v8"
)

func main() {
    rdb := redis.NewClient(&redis.Options{
        Addr: "localhost:6379",
    })

    ctx := context.Background()
    cmdsRes := rdb.Command(ctx)

    var readonly []string

    cc, err := cmdsRes.Result()
    if err != nil {
        log.Fatal(err)
    }
    for _, r := range cc {
        if !r.ReadOnly {
            continue
        }

        readonly = append(readonly, r.Name)
    }

    fmt.Println(readonly)
}

This procedure demonstrates how to prove that using the go-transaction-manager (TRM) does not prevent you from updating underlying drivers (like pgx/v5) to newer versions.

Follow these steps to verify compatibility:

1. Install the pgxv5 driver with an older pgx version: Use the specific TRM driver version and disable GOWORK to pin the dependency.
2. Verify pinning: Check go.mod to ensure github.com/jackc/pgx/v5 is locked to the older version (e.g., v5.5.1).
3. Prepare environment: Run go mod tidy and go mod vendor to finalize the old versions.
4. Perform update: Manually update pgx using go get or go mod tidy and verify in go.mod that it has moved to the latest version (e.g., v5.6.0).
5. Run tests: Execute the verification tests to ensure the system remains stable with the new driver version.

# 1. Install pgxv5 driver with old pgx version
GOWORK=off go get github.com/avito-tech/go-transaction-manager/drivers/pgxv5/v2@v2.0.0-rc9.2

# 3. Install the old versions
GOWORK=off go mod tidy && GOWORK=off go mod vendor

# 4. Update pgx manually
go get github.com/jackc/pgx/v5
# or
go mod tidy

# 5. Run verification tests
go test ./...

The pgxv4.Transaction type implements the trm.Transaction interface for pgx.Tx. It manages the lifecycle of a PostgreSQL transaction using the pgx/v4 driver.

Concurrency Warning: Transaction is NOT safe for concurrent use. Because pgx.Tx does not support running commands from multiple goroutines simultaneously, you must ensure that a query never overlaps with a Commit or Rollback call on the same transaction.

Lifecycle Note: Context cancellation does not automatically roll back the transaction from a background goroutine. When using manager.Manager, the rollback is issued after the transactional function returns. If calling standalone, you must explicitly call Rollback yourself.

// Example of how a Transaction object is structured conceptually
// It wraps a pgx.Tx and tracks its closed state.
type Transaction struct {
	tx       pgx.Tx
	isClosed *drivers.IsClosed
}

To manage transactions, create a factory using a driver (e.g., trmsqlx.NewDefaultFactory(db)), initialize a manager with manager.Must(), and use trManager.Do(ctx, func(ctx context.Context) error { ... }) to wrap your database operations.

To access the transaction within your repository, use the driver's CtxGetter (e.g., trmsqlx.DefaultCtxGetter) to retrieve either the active transaction or the base database connection via DefaultTrOrDB(ctx, db).

package main

import (
	"context"
	"fmt"

	"github.com/jmoiron/sqlx"
	_ "github.com/mattn/go-sqlite3"

	trmsqlx "github.com/avito-tech/go-transaction-manager/drivers/sqlx/v2"
	"github.com/avito-tech/go-transaction-manager/trm/v2/manager"
)

func main() {
	db, err := sqlx.Open("sqlite3", "file:test?mode=memory")
	checkErr(err)
	defer db.Close()

	sqlStmt := `CREATE TABLE IF NOT EXISTS user (user_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, username TEXT);`
	_, err = db.Exec(sqlStmt)
	checkErr(err, sqlStmt)

	r := newRepo(db, trmsqlx.DefaultCtxGetter)
	ctx := context.Background()
	trManager := manager.Must(trmsqlx.NewDefaultFactory(db))
	u := &user{Username: "username"}

	err = trManager.Do(ctx, func(ctx context.Context) error {
		checkErr(r.Save(ctx, u))

		// example of nested transactions
		return trManager.Do(ctx, func(ctx context.Context) error {
			u.Username = "new_username"
			return r.Save(ctx, u)
		})
	})
	checkErr(err)

	userFromDB, err := r.GetByID(ctx, u.ID)
	checkErr(err)

	fmt.Println(userFromDB)
}

func checkErr(err error, args ...interface{}) {
	if err != nil {
		panic(fmt.Sprint(append([]interface{}{err}, args...)...))
	}
}

type repo struct {
	db     *sqlx.DB
	getter *trmsqlx.CtxGetter
}

func newRepo(db *sqlx.DB, c *trmsqlx.CtxGetter) *repo {
	return &repo{db: db, getter: c}
}

type user struct {
	ID       int64  `db:"user_id"`
	Username string `db:"username"`
}

func (r *repo) GetByID(ctx context.Context, id int64) (*user, error) {
	query := "SELECT * FROM user WHERE user_id = ?;"
	u := user{}

	return &u, r.getter.DefaultTrOrDB(ctx, r.db).GetContext(ctx, &u, r.db.Rebind(query), id)
}

func (r *repo) Save(ctx context.Context, u *user) error {
	query := `UPDATE user SET username = :username WHERE user_id = :user_id;`
	if u.ID == 0 {
		query = `INSERT INTO user (username) VALUES (:username);`
	}

	res, err := sqlx.NamedExecContext(ctx, r.getter.DefaultTrOrDB(ctx, r.db), r.db.Rebind(query), u)
	if err != nil {
		return err
	} else if u.ID != 0 {
		return nil
	} else if u.ID, err = res.LastInsertId(); err != nil {
		return err
	}

	return err
}

The following regex patterns are used during the codegen process to transform raw Redis command lists into structured Go code for the transaction manager.

Command Wrapper Pattern

Find: `^(⁽+)(((?:((?:, )?\w+)(?: (?:^,)+))?)?(?:((?:, )?\w+)(?: (?:^,)+))?)?(?:((?:, )?\w+)(?: (?:^,)+))?)?(?:((?:, )?\w+)(?: (?:^,)+))?)?(?:((?:, )?\w+)(?: (?:^,)+))?)?)) (*?)(.+)

Replace:

func(p *WritePipeliner) $1$2 $8redis.$9 {
	return p.read.$1($3$4$5$6$7)
}

Argument Prefixing

To ensure arguments are correctly typed for the Redis driver, use these patterns:

Set redis for arguments:

• Find: (\w)\((.*\w )\*(\w+.*)\)
• Replace: $1($2*redis.$3)

Set ellipsis for arguments:

• Find: (keys|members|pos|fields)\)
• Replace: $1...)"

Find: `^([^(]+)(\((?:((?:, )?\w+)(?: (?:[^,)]+))?)?(?:((?:, )?\w+)(?: (?:[^,)]+))?)?(?:((?:, )?\w+)(?: (?:[^,)]+))?)?(?:((?:, )?\w+)(?: (?:[^,)]+))?)?(?:((?:, )?\w+)(?: (?:[^,)]+))?)?\)) (\*?)(.+)

Replace:
```regexp
func(p *WritePipeliner) $1$2 $8redis.$9 {
	return p.read.$1($3$4$5$6$7)
}

Find: (\w)\((.*\w )\*(\w+.*)\) replace: $1($2*redis.$3)

Find: (keys|members|pos|fields)\) replace: $1...)"

Benchmarks comparing the performance of the Go Transaction Manager (TRM) against a clean implementation (without TRM) across different database drivers show the overhead introduced by the manager.

Summary of Findings

• Overhead: TRM decreases performance by approximately 11% on average.
• Context: The overhead is significantly smaller than the time taken by filesystem operations or network connections.

Benchmark Results by Driver

SQLite in Memory

• Average Overhead: ~17.7% (Diff: ~5241 ns)
• Range: 12.4% to 21.2%

SQLite in File

• Average Overhead: ~0.68% (Diff: ~4096 ns)
• Range: -4.3% to 10.6%

sqlmock

• Average Overhead: ~3.4% (Diff: ~49971 ns)
• Range: 0.67% to 9.05%

MockDB

• BenchmarkClean_MockDB-12: 219,069 ns/op
• BenchmarkTRM_MockDB-12: 219,373 ns/op

| trm  (ns) | clean (ns) | diff (ns) | percent     |
|-----------|------------|-----------| ----------- |
| 35072     | 29731      | 5341      | 17,96441425 |
| 34787     | 29422      | 5365      | 18,23465434 |
| 35060     | 29376      | 5684      | 19,34912854 |