msgp Documentation

By default, the code generator implements several interfaces to allow for different serialization patterns:

• msgp.Sizer: For determining the size of the encoded data.
• msgp.Encodable and msgp.Decodable: Optimized for stream serialization using *msgp.Writer and *msgp.Reader (which are protocol-aware versions of *bufio.Writer and *bufio.Reader).
• msgp.Marshaler and msgp.Unmarshaler: Similar to json.Marshaler and json.Unmarshaler, typically used for []byte-oriented operations.

Carefully-designed applications can use these methods to achieve marshalling/unmarshalling with zero heap allocations.

To trigger code generation for a Go file, include the //go:generate msgp directive in your source file. The msgp command will then generate serialization methods for all exported type declarations found in that file.

Important: msgp operates on individual files. If your structs include types defined in other files, those files must also be processed by the generator.

To use the code generation features, you must first install the msgp command-line tool using Go:

go install github.com/tinylib/msgp@latest

The msgp generator can produce two variants of set types depending on whether element order is required:

1. Unsorted Sets (e.g., Foo): Elements are stored in a map and encoded as a MessagePack array. The order of elements in the encoded output is not guaranteed.
2. Sorted Sets (e.g., FooSorted): Elements are stored in a map, but during EncodeMsg, MarshalMsg, AsSlice, and MarshalJSON, the elements are sorted before processing. This ensures a deterministic output order.

The msgp tool is designed to be used with the go generate command. To automate the generation of MessagePack serialization methods for your Go types, add the following directive to your source files:

//go:generate msgp

When running go generate, the tool automatically detects the current file via the $GOFILE environment variable and uses appropriate defaults for output and method generation.

You can control how field names are encoded in MessagePack using struct tags, following a pattern similar to the encoding/json package. Use the msg key to specify the name. Use msg:"-" to ignore a field, and note that unexported fields are ignored by default.

type Person struct {
	Name       string `msg:"name"`    // Encoded as "name"
	Address    string `msg:"address"` // Encoded as "address"
	Age        int    `msg:"age"`     // Encoded as "age"
	Hidden     string `msg:"-"`       // This field is ignored
	unexported bool             // This field is also ignored
}

type Person struct {
	Name       string `msg:"name"`
	Address    string `msg:"address"`
	Age        int    `msg:"age"`
	Hidden     string `msg:"-"` // this field is ignored
	unexported bool             // this field is also ignored
}

When using msgp, be aware of the following constraints:

• Map Keys: Maps must have string keys to preserve JSON interoperability. While the deserializer allows reading map keys encoded as bin types, they will be cast to Go strings.
• Ignored Fields: chan fields, func fields, and non-exported fields are ignored.
• Interface{} Encoding: Encoding of interface{} is limited to built-in types or types that have explicit encoding methods.
• External Identifiers: Identifiers from outside the processed source file are assumed to satisfy the generator's interfaces; if they do not, the code will fail to compile.

Generated set types implement the standard json.Marshaler and json.Unmarshaler interfaces. Sets are represented as JSON arrays (e.g., ["val1", "val2"]).

• MarshalJSON() ([]byte, error): Encodes the set as a JSON array. If the set is nil, it encodes to null.
• UnmarshalJSON(data []byte) error: Decodes a JSON array into the set. If the data is null, the set becomes nil.

The msgp code generator provides optimized methods for serializing and deserializing custom set types (represented as map[T]struct{}) to and from MessagePack.

For a generated set type (e.g., Foo), you can use the following methods:

• EncodeMsg(writer *msgp.Writer) error: Encodes the set into the provided MessagePack writer. If the set is nil, it writes a MessagePack nil.
• MarshalMsg(bytes []byte) ([]byte, error): Encodes the set into a byte slice. This is useful for buffer-based serialization.
• DecodeMsg(reader *msgp.Reader) error: Decodes the set from the provided MessagePack reader.
• UnmarshalMsg(bytes []byte) ([]byte, error): Decodes the set from a byte slice.
• Msgsize() int: Returns the maximum size required to encode the message, useful for pre-allocating buffers.

If you need to trigger code generation programmatically rather than via the CLI, you can use the Run function.

Signature: func Run(gofile string, mode gen.Method, unexported bool) error

• gofile: The path to the input file.
• mode: A bitmask of gen.Method defining which methods to generate (e.g., gen.Encode, gen.Decode, gen.Marshal, gen.Unmarshal, gen.Size, or gen.Test).
• unexported: A boolean indicating whether to process unexported types and fields.

Generated set types include utility methods to convert between the set representation (map[T]struct{}) and standard Go slices ([]T).

• AsSlice() []T: Returns a slice containing all elements of the set. If the set is unsorted, the order is not guaranteed. If the set is a Sorted type, the returned slice is sorted.
• [TypeName]FromSlice(s []T) [TypeName]: A constructor that creates a new set from an existing slice. If the input slice is nil, it returns nil.

You can invoke the msgp command-line tool directly with various flags to customize the code generation process.

Available Flags:

• -o: Specify the output file name (defaults to {input}_gen.go).
• -file: Specify the input file name or directory (defaults to the $GOFILE environment variable).
• -io: Whether to satisfy the msgp.Decodable and msgp.Encodable interfaces (default: true).
• -marshal: Whether to satisfy the msgp.Marshaler and msgp.Unmarshaler interfaces (default: true).
• -tests: Whether to generate tests and benchmarks (default: true).
• -unexported: Process unexported types and fields (default: false).
• -v: Enable verbose diagnostics.
• -d: Apply a directive to all files. Multiple -d flags are allowed. You can use the prefix msgp: or omit it.