Go Text (x/text)

The repository allows updating the underlying Unicode or CLDR data versions using environment variables with go generate.

Important Notes:

• Backwards compatibility is not maintained when updating versions; the code is adapted to the new data, which may cause breaking changes.
• UNICODE_VERSION must correspond to a directory in https://www.unicode.org/Public/.
• CLDR_VERSION must correspond to a directory in https://www.unicode.org/Public/cldr/.
• Updating to a Unicode version newer than the one in core will also update relevant packages in core. The idna package in x/net is always updated.

# To update a Unicode version
UNICODE_VERSION=x.x.x go generate

# To update a CLDR version
CLDR_VERSION=version go generate

To run all tests in the repository, use the standard Go test command from the root directory. If you need to run ICU conformance tests, you must have the correct ICU version installed on your system and include the -tags icu flag.

go test ./...
go test -tags icu ./...

To generate the tables used by the packages (excluding encoding tables), run go generate from the repository root. By default, this uses the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.

Running this command will create a DATA subdirectory which acts as both a source for the tables and a cache.

The gotext tool is used for managing translations and text within Go projects. It provides a workflow for extracting strings, rewriting code to support localization, generating translation code, and merging translations into a catalog.

Available commands:

• update: Merges translations and generates a catalog.
• extract: Extracts strings from your code that need to be translated.
• rewrite: Rewrites fmt function calls to use a message.Printer instead.
• generate: Generates the Go code required to insert translated messages into your application.

gotext command [arguments]

The gotext tool is used for managing text and translations in Go source code. It provides subcommands to extract, rewrite, and generate translation files.

Basic usage:

gotext <command> [arguments]

Available commands include:

• update (full-cycle update of extraction, sending, and integration)
• extract (extracts messages from source code)
• rewrite (rewrites messages)
• generate (generates translation files)

To see all available commands, run gotext without arguments or gotext help. To see details about a specific command, use gotext help <command>.

gotext extract

The gotext tool accepts several global flags that configure how the translation pipeline operates. These flags affect all subcommands.

Note: -srclang accepts multiple tags separated by commas (e.g., -srclang=en-US,fr-FR).

gotext -srclang=en-US,de-DE -dir=translations extract

The generate command supports the following flag:

-out <gofile>  output file to write to

Use the extract command to scan your Go packages and identify strings that should be marked for translation.

Usage: gotext extract <package>*

The rewrite command is a tool used to automate the migration of a project from standard fmt functions to using x/text's message.Printer. This is typically a one-time operation for a project.

What it does:

• Rewrites usages of fmt to use a message.Printer whenever one is in scope.
• Converts Print and Println calls that use constant strings into Printf calls. This allows translators to reorder arguments, which is essential for localization.

Usage: rewrite <package>

Flags:

• -w: Write files in place (if not provided, output is sent to os.Stdout).

# To see the changes in stdout without modifying files:
rewrite ./my/package

# To apply the changes directly to the files in the package:
rewrite -w ./my/package

The generate command is used to scan specified Go packages and generate the necessary code to insert translated messages into your application. It performs extraction, importing, and merging of message data before final generation.

Usage: generate <package> [-out <gofile>]

Arguments:

• <package>: The Go package(s) to scan for translatable messages.
• -out <gofile>: (Optional) Specifies the output file where the generated code will be written.

gotext generate <package> [-out <gofile>]

Use the generate command to produce the Go source code required to actually insert translated messages into your application.

Usage: gotext generate <package> [-out <gofile>]

The extract command identifies and extracts strings intended for translation from the specified Go packages. It processes the provided packages and performs a sequence of operations: extraction, importing, merging, and exporting the results.

Usage: extract <package>*

Flags:

• --lang: A comma-separated list of languages to process. Defaults to en-US.

Workflow:

1. Extract: Scans the provided packages for translatable strings.
2. Import: Loads existing translation data.
3. Merge: Combines extracted strings with existing data.
4. Export: Saves the resulting translation state to the output format.

# Example usage (placeholders used for package names)
gotext extract ./pkg/myapp/...,--lang=en-US,fr-FR