go-internal

The go-internal repository provides an opinionated selection of internal packages and testing utilities extracted from the Go standard library. It is primarily used by Go contributors and projects like CUE that require specialized testing infrastructure or Go-specific internal logic.

Available packages include:

• goproxytest: A GOPROXY implementation designed for testing.
• gotooltest: Utilities for using the Go tool within test scripts.
• imports: Support for reading import statements and lists of known architectures and OSs.
• par: Utilities for performing work in parallel.
• testenv: Provides information about the current testing environment.
• testscript: A shell-like test environment for testing Go CLI commands.
• txtar: Simple text-based file archives for testing (Note: users should prefer golang.org/x/tools/txtar where possible).

You can simulate Go module proxy behavior by including a .gomodproxy subdirectory in your testscript file. Files and directories within .gomodproxy are served via a goproxytest server, which is made available to each script through the GOPROXY environment variable.

Note: The contents of the .gomodproxy directory are not directly accessible to the script; they can only be accessed via the proxy server. Refer to github.com/rogpeppe/go-internal/goproxytest for the specific file format requirements.

# Example structure within a .txtar file

-- go.mod --
module mod

-- .gomodproxy/fruit.com_v1.0.0/.mod --
module fruit.com

-- .gomodproxy/fruit.com_v1.0.0/.info --
{"Version":"v1.0.0","Time":"2018-10-22T18:45:39Z"}

-- .gomodproxy/fruit.com_v1.0.0/fruit/fruit.go --
package fruit

const Name = "Apple"

The testscript package provides a shell-like environment specifically tuned for testing Go CLI commands. It is extracted from the Go core team's internal testing infrastructure used to test the go command itself.

Key Features:

• CLI Assertions: Supports patterns for checking stderr/stdout and command pass/fail assertions.
• Go Integration: Integrates with go test and supports Go-specific concepts like build tags.
• Golden Files: Supports automatic updating of golden files via testscript.Params.
• Archive Support: Uses the txtar text archive format for input and sample output files.
• Standalone Execution: Includes a testscript command for running standalone scripts with files embedded in txtar format.

To create a testscript plugin, you must implement the Interface type. While the full definition of Interface is not in this file, the dispatch logic reveals the required capabilities:

• Info() InfoResult: Returns metadata about the plugin (including its version).
• NewTestInstance(p TestParams) (TestInstance, error): Creates a new isolated test environment.
• Close(): Cleans up the plugin implementation.

Additionally, the TestInstance returned by NewTestInstance must implement:

• Env() Env: Returns the environment information for the instance.
• RunCmd(params []string) (json.RawMessage, error): Executes a command within that specific instance.
• Close(): Cleans up the specific test instance.

The txtar-goproxy command runs a Go module proxy from a txtar module directory (typically created by the txtar-addmod command). This is useful for testing how Go commands interact with a proxy by serving a local set of modules.

To use it, run the command pointing to your module directory, then set your GOPROXY environment variable to the URL provided by the tool.

# 1. Start the proxy pointing to your txtar directory
./txtar-goproxy ./path/to/txtar/dir

# 2. Use the URL printed by the command (e.g., http://localhost:1234/mod)
export GOPROXY=http://localhost:1234/mod

# 3. Run your Go commands
go test -proxy=localhost:1234

This example demonstrates how to use testscript to verify the behavior of the goimports tool, including installing the tool and checking the output of a diff.

go install golang.org/x/tools/cmd/goimports

# check goimports help information
exec goimports -d main.go
stdout 'import "math"'

-- go.mod --
module mod

require golang.org/x/tools v0.0.0-20181221235234-d00ac6d27372

-- main.go --
package mod

const Pi = math.Pi

This example demonstrates how to use a .txtar file to simulate fetching a module from a proxy and verifying the output.

go get -m fruit.com
go list fruit.com/...
stdout 'fruit.com/fruit'

-- go.mod --
module mod

-- .gomodproxy/fruit.com_v1.0.0/.mod --
module fruit.com

-- .gomodproxy/fruit.com_v1.0.0/.info --
{"Version":"v1.0.0","Time":"2018-10-22T18:45:39Z"}

-- .gomodproxy/fruit.com_v1.0.0/fruit/fruit.go --
package fruit

const Name = "Apple"

Use the following flags to control the behavior of the testscript command:

• -v: Enables verbose output.
• -e VAR[=value]: Passes environment variables through to each script. You can use this flag multiple times to set multiple variables. Note that VAR overrides testscript-defined values, except for WORK, which cannot be overridden.
• -u: Enables script updating. If a cmp command within a script fails and its second argument refers to a file inside the testscript file, the command will succeed and the testscript file will be updated to reflect the actual content. This is the CLI equivalent of testscript.Params.UpdateScripts.
• -work: Prints the path to the temporary work directory before running each script and prevents the directory from being removed when testscript exits.

This example demonstrates a reproduction script (fruit.txtar) that uses the .gomodproxy feature to simulate a module download.

go get -m fruit.com
    go list fruit.com/...
    stdout 'fruit.com/fruit'

-- go.mod --
    module mod

-- .gomodproxy/fruit.com_v1.0.0/.mod --
    module fruit.com

-- .gomodproxy/fruit.com_v1.0.0/.info --
    {"Version":"v1.0.0","Time":"2018-10-22T18:45:39Z"}

-- .gomodproxy/fruit.com_v1.0.0/fruit/fruit.go --
    package fruit

const Name = "Apple"

This example shows a script (goimports.txtar) that installs a tool and verifies its output using exec and stdout checks.

go install golang.org/x/tools/cmd/goimports

# check goimports help information
    exec goimports -d main.go
    stdout 'import "math"'

-- go.mod --
    module mod

require golang.org/x/tools v0.0.0-20181221235234-d00ac6d27372

-- main.go --
    package mod

const Pi = math.Pi

The testscript command is used to run github.com/rogpeppe/go-internal/testscript scripts within a fresh, temporary work directory. This allows you to create self-contained reproductions of command sequences. Each file provided as an argument is treated as a script. The special filename - is interpreted as standard input.

testscript [-v] [-e VAR[=value]]... [-u] [-work] files...