thriftgo

repository·main·Indexed 43 days ago

https://github.com/cloudwego/thriftgo

An Apache Thrift IDL compiler written in Go that generates Go source code from .thrift files. It features an extensible plugin system, a high-performance fastgo backend, and specific optimizations for the CloudWeGo ecosystem. It supports advanced features such as field-masking for selective serialization, Thrift streaming, and the ability to import types from existing packages via code_ref to avoid duplicate definitions.

Tokens
6.6K
Snippets
20
Records
30
Agent score
5%

What's inside thriftgo

  1. What definitions are always kept by trimmer

    main

    The following elements are never removed, regardless of reachability or flags:

    • Constants — all constants are kept unconditionally.
    • Typedefs — all typedefs are kept unconditionally.
    • Enums — all enums are kept unconditionally.
    • Included files containing constants, enums, or typedefs — an include statement is retained (and its IDL recursively trimmed) even if no struct types from it are directly referenced.
    • @preserve-annotated structs — any struct, union, or exception whose reserved comments contain a line matching // @preserve or # @preserve (case-insensitive) is kept, unless -p false is passed.
  2. How string literals are handled in the IDL

    main

    Thriftgo supports two forms of string literals in the IDL: single-quoted ('...') and double-quoted ("..."). These literals are used for include declarations, string constant initializations, and annotations.

    The Unescaping Rule: Thriftgo copies literals to the generated code with only the delimiters used to quote the string unescaped. Other escape sequences (like \t, \n, or Unicode escapes like \u65b0) are preserved as-is so that the target language (e.g., Go) can interpret them during runtime.

    Key Behaviors:

    • You do not need to escape a single quote inside a double-quoted literal, or vice versa.
    • Escape sequences that do not conflict with the chosen delimiter are allowed.
    • For compatibility with existing IDLs, the Go generator allows escaped double quotes within a quoted literal by default.
    // Example IDL usage
    const string str = "'double'\t\"quoted\""
    
    struct S {
        1: string f1 = 'single\'"quoted' (go.tag = "json:\"hello\tworld\"")
    }
  3. Handle Apache Thrift compatibility and dependencies

    main

    Thriftgo generates code that depends on github.com/apache/thrift v0.13.0.

    Warning: Version 0.14.0+ is incompatible because it introduces breaking changes to TProtocol, TTransport, and TProcessor by adding context.Context parameters.

    To manage dependencies:

    1. Avoid Apache Thrift dependency entirely: Use no_default_serdes and no_processor together. This generates plain Go structs without Read/Write methods or processor/client code.
    2. Redirect imports: Use thrift_import_path=<path> to point to a specific fork or a vendored copy of the Thrift runtime.
  4. Import types from existing packages using code_ref

    main

    When multiple services share types from a common IDL, you can avoid duplicate definitions by importing types from an already-generated Go package instead of regenerating them. This is controlled by the code_ref family of flags.

    Setup: You must create an idl-ref.yaml (or idl-ref.yml) in your working directory that maps IDL paths to existing Go package paths:

    ref:
      path/to/shared.thrift: github.com/myorg/shared/gen

    Options:

    • code_ref: Replaces local generation with imports. Generates a <name>-ref.go file containing type aliases.
    • code_ref_slim: Generates fewer aliases to reduce import conflicts.
    • exp_code_ref: (Experimental) Similar to slim but keeps some structs as local definitions.
    • keep_code_ref_name: Uses the original filename instead of appending -ref.go.
  5. Keep only specific methods with the -m flag

    main

    By default, trimmer keeps every service method and its dependencies. Using the -m flag allows you to restrict the output to only specific methods and their required types. The service block remains in the output but will only contain the specified methods.

    Format: ServiceName.MethodName (supports regexp2 patterns).

    Prefix Rules:

    • If no ServiceName. prefix is provided in a single-service IDL, the service name is prepended automatically.
    • If no ServiceName. prefix is provided in a multi-service IDL, the last service name in the file is used.

    Example (specifying multiple methods):

    trimmer -m UserService.GetUser -m UserService.DeleteUser service.thrift
  6. Recursively trim an IDL tree with the -r flag

    main

    The -r flag enables recursive mode. When used, trimmer walks all transitive include statements and dumps each one as well, preserving the relative directory structure.

    Requirements:

    1. The -r value must be the base directory used to compute relative output paths.
    2. The -o (output) directory must be outside the -r directory to avoid overwriting source files.
    3. If -o is not specified when using -r, it defaults to ./trimmed_idl.

    Example:

    # If idl/service.thrift includes idl/base/base.thrift
    # Output: trimmed_idl/service.thrift, trimmed_idl/base/base.thrift
    trimmer -r ./idl -o ./trimmed_idl ./idl/service.thrift
    trimmer -r ./idl -o ./trimmed_idl ./idl/service.thrift
  7. Enable Field-Mask support for selective serialization

    main

    To enable field-mask support (similar to Protobuf FieldMask) for selectively serializing/deserializing struct fields, you must provide a specific combination of flags in the Go backend options.

    Required flags:

    • with_reflection: Generates *-reflection.go files with type descriptors. This is required for path validation.
    • with_field_mask: Adds a _fieldmask *fieldmask.FieldMask field and Set_FieldMask() method to structs. Read/Write methods will then respect this mask.

    Optional modifiers:

    • field_mask_halfway: Allows setting the mask on nested structs, not just the root.
    • field_mask_zero_required: If a required field is excluded by a mask, it is written as a zero value instead of its current value.
    # Minimal usage for field-mask support
    thriftgo -g go:with_field_mask,with_reflection service.thrift
  8. Enable Thrift streaming support

    main

    By default, service methods with streaming annotations are dropped from the generated output. To enable streaming, use the following options:

    1. thrift_streaming: Enables generation of streaming methods.
    2. streamx (Optional): Changes the generated streaming interface types to use the github.com/cloudwego/kitex/pkg/streaming (streamx) API. This requires thrift_streaming to be enabled.

    Use thrift_streaming alone for the default Kitex streaming API, or combine it with streamx for the newer API.

    # Example: Generate with streamx support
    thriftgo -g go:thrift_streaming,streamx service.thrift
  9. Install the trimmer tool

    main

    The trimmer tool is a standalone utility to remove unused definitions from Thrift IDL files. It keeps only definitions reachable from service method signatures.

    Prerequisites: Go 1.18 or later. Ensure $GOPATH/bin (default: ~/go/bin) is in your PATH.

    To install the latest version via Go:

    go install github.com/cloudwego/thriftgo/tool/trimmer@latest
  10. Configure Go backend options

    main

    Go-specific options are passed via the -g go:<options> flag. Options are a comma-separated list. Boolean options can be true, false, or empty (which defaults to true).

    Example usage:

    thriftgo -g go:naming_style=golint,gen_setter,template=slim example.thrift