ts-proto

repository·main·Indexed 25 days ago

https://github.com/stephenh/ts-proto

A code generator that transforms Protobuf (.proto) files into idiomatic, strongly-typed TypeScript interfaces and service clients. It produces clean, TypeScript-first code treating messages as plain data structures. Version 2.12.0 supports integration with protoc, Buf, and Gradle, and provides specialized options for NestJS gRPC microservices, ESM environments, and Twirp client auto-batching.

Tokens
9.2K
Snippets
22
Records
51
Agent score
83%

What's inside ts-proto

  1. Understand default values and unset fields

    main

    Following Protobuf rules, ts-proto cannot distinguish between a field being explicitly set to its default value and a field being unset.

    • Decoding: ts-proto always returns default values for unset fields (e.g., '' for string, 0 for number).
    • Encoding: ts-proto omits unset fields and fields set to their default values from the binary output.
    • JSON (fromJSON / toJSON):
      • Use fromJSON to normalize input, as it will initialize default values for missing fields.
      • toJSON normalizes messages by omitting unset fields and fields set to their default values.

    If you need to detect if a primitive field was actually set, use Wrapper Types.

    syntax = "proto3";
    message Foo {
      string bar = 1;
    }
    // Decoding an empty buffer
    Foo.decode(protobufBytes); // => { bar: '' }
    
    // Encoding a default value
    Foo.encode({ bar: "" }); // => { }, writes an empty Foo object
    
    // JSON Normalization
    Foo.fromJSON({}); // => { bar: '' }
    Foo.toJSON({ bar: "" }); // => { }
  2. How NestJS service interfaces are named

    main

    When using nestJs=true, ts-proto generates two distinct TypeScript interfaces for every service defined in your .proto file:

    1. Controller Interface: Used to implement the service logic in your NestJS controller. It is named by appending Controller to the service name (e.g., HeroService becomes HeroServiceController).
    2. Client Interface: Used by the client code to call the service. It is named by appending Client to the service name (e.g., HeroService becomes HeroServiceClient).

    Additionally, ts-proto generates constants for the package name (e.g., HERO_PACKAGE_NAME) and the service name (e.g., HERO_SERVICE_NAME) to ensure your code breaks at compile-time if the proto definitions change, rather than failing at runtime.

    service HeroService {
        rpc FindOneHero (HeroById) returns (Hero) {}
    }
    // Generates: HeroServiceController (for server) and HeroServiceClient (for client)
  3. How Wrapper Types handle optional primitives

    main

    Wrapper Types are messages containing a single primitive field (from google/protobuf/wrappers.proto). Because they are messages, their default value is undefined, which allows you to distinguish between an unset field and a field set to its default primitive value (e.g., 0 or "").

    ts-proto generates these fields as <primitive> | undefined.

    • Encoding: The primitive value is converted back to the corresponding wrapper type (e.g., string becomes { value: 'foo' } in binary).
    • JSON: When calling .toJSON(), the value is not converted to a wrapper object; it remains an idiomatic JSON primitive.
    syntax = "proto3";
    
    import "google/protobuf/wrappers.proto";
    
    message ExampleMessage {
      google.protobuf.StringValue name = 1;
    }
    interface ExampleMessage {
      name: string | undefined;
    }
    ExampleMessage.encode({ name: "foo" }); // => { name: { value: 'foo' } }, in binary
    ExampleMessage.toJSON({ name: "foo" }); // => { name: 'foo' }
  4. Implement the Rpc interface for custom transport

    main

    ts-proto is RPC framework agnostic. All generated client implementations expect an Rpc object that satisfies the following interface:

    interface Rpc {
      request(service: string, method: string, data: Uint8Array): Promise<Uint8Array>;
    }

    You must provide an implementation of this request method that handles the actual network transmission (e.g., via gRPC, HTTP, etc.).

  5. Understand the ts-proto generated code structure

    main

    Unlike protobufjs which often uses classes, ts-proto generates interfaces for messages. This allows you to treat messages as plain data structures (hashes).

    Each message also includes a companion object containing factory and serialization methods:

    • create(baseObject?): Creates a new instance.
    • encode(message, writer): Serializes the message to bytes.
    • decode(reader): Deserializes bytes into the message interface.
    • fromJSON(object): Creates a message from a JSON object.
    • toJSON(message): Converts the message to a JSON-compatible object (using proto3 canonical JSON encoding).
    • fromPartial(object): Creates a message from a partial object (useful for construction).
    export interface Simple {
      name: string;
      age: number;
      createdAt: Date | undefined;
      child: Child | undefined;
      state: StateEnum;
      grandChildren: Child[];
      coins: number[];
    }
    
    export const Simple = {
      create(baseObject?: DeepPartial<Simple>): Simple { ... },
      encode(message: Simple, writer: Writer = Writer.create()): Writer { ... },
      decode(reader: Reader, length?: number): Simple { ... },
      fromJSON(object: any): Simple { ... },
      fromPartial(object: DeepPartial<Simple>): Simple { ... },
      toJSON(message: Simple): unknown { ... },
    };
  6. How `oneof` fields are handled in TypeScript

    main

    By default, ts-proto models oneof fields "flatly". This means every field in the oneof becomes an optional property on the message type. This requires manual checks to ensure only one is set and manual unsetting of others.

    To avoid this, it is highly recommended to use the oneof=unions-value option. This generates an Algebraic Data Type (ADT) where the oneof is represented by a single field containing a discriminated union. This approach automatically enforces that only one case is set at a time.

    There is also a oneof=unions option which generates a union where field names are included in each option, but this is no longer recommended as it is more difficult to handle in code.

  7. How JSON Struct Types are handled

    main

    Protobuf's standard types cannot represent arbitrary JSON values. To handle this, you can use Struct Types from google/protobuf/struct.proto. ts-proto automatically converts between these Protobuf types and their corresponding TypeScript/JSON representations.

    • google.protobuf.Value maps to any (represents number | string | boolean | null | array | object).
    • google.protobuf.ListValue maps to any[] (represents a JSON array).
    • google.protobuf.Struct maps to { [key: string]: any } (represents a JSON object).

    When encoding a JSON value into a message, ts-proto converts it into the appropriate Protobuf Struct structure.

    syntax = "proto3";
    
    import "google/protobuf/struct.proto";
    
    message ExampleMessage {
      google.protobuf.Value anything = 1;
    }
    interface ExampleMessage {
      anything: any | undefined;
    }
    ExampleMessage.encode({ anything: { name: "hello" } });
    // Outputs a Value containing a Struct with a MapEntry for 'name'
    
    ExampleMessage.encode({ anything: true });
    // Outputs a Value containing a boolValue
  8. How Auto-Batching and N+1 Prevention works

    main

    For Twirp clients, ts-proto can automatically batch individual RPC calls into a single batch call to prevent the N+1 problem.

    Requirements for the Backend

    To enable this, your service must implement a batching convention:

    1. Method Name: Must follow the pattern Batch<OperationName> (e.g., BatchGetBook).
    2. Input Type: Must have a single repeated field (e.g., repeated string ids = 1).
    3. Output Type: Must return either:
      • A single repeated field where the order matches the input IDs.
      • A map<string, Entity> mapping the input ID to the output entity.

    Requirements for the Client

    1. Build Parameter: You must enable useContext=true during code generation. This adds a ctx parameter to client methods and provides access to getDataLoaders for request-scoped caching.
    2. Client Usage: ts-proto will generate a non-batch version of the method (e.g., client.GetBook(id)). The client code can call this single-item method, but the underlying implementation will automatically batch these calls into the BatchGetBook RPC.
  9. Summary of Optional and Required value patterns

    main

    When designing your .proto files for use with ts-proto, follow these patterns to manage optionality:

    • Required primitives: Use as-is (e.g., string name = 1;).
    • Optional primitives: Use Wrapper Types (e.g., google.protobuf.StringValue name = 1;).
    • Required messages: Not available in proto3 (messages are inherently optional/nullable).
    • Optional messages: Use as-is (e.g., SubMessage message = 1;).
  10. Configure ts-proto with Buf

    main

    If you use Buf for managing your protobuf files, you can integrate ts-proto in two ways:

    1. Using a local plugin

    In your buf.gen.yaml, set strategy: all to ensure all files are processed:

    version: v1
    plugins:
      - name: ts
        out: ../gen/ts
        strategy: all
        path: ../node_modules/ts-proto/protoc-gen-ts_proto

    To prevent buf push from including unnecessary files, add node_modules to your buf.yaml excludes:

    build:
      excludes: [node_modules]

    2. Using the official Buf Registry plugin

    You can use the community-published plugin directly:

    version: v1
    plugins:
      - plugin: buf.build/community/stephenh-ts-proto
        out: ../gen/ts
        opt:
          - outputServices=...
          - useExactTypes=...