Twirp RPC Framework

repository·main·Indexed 11 days ago

https://github.com/twitchtv/twirp

A lightweight RPC framework for service-to-service communication that generates routing and serialization code from Protobuf API definitions. Built on the standard Go net/http library, Twirp supports HTTP 1.1 and JSON serialization, providing a minimalist alternative to gRPC.

Tokens
18.3K
Snippets
57
Records
86
Agent score
84%

What's inside Twirp

  1. What is Twirp?

    main

    Twirp is a minimalist framework for service-to-service communication. It generates routing and serialization from API definition files (Protobuf), allowing developers to focus on application logic rather than HTTP methods, paths, or JSON handling.

    Key characteristics:

    • Standard Library Based: Unlike gRPC, Twirp runs on the standard Go net/http Server.
    • Protocol Support: It can run on HTTP 1.1 (not just HTTP/2).
    • Serialization: Supports JSON serialization, which simplifies debugging.
    • Automation: Provides autogenerated clients and a structured framework for error handling.
  2. Use gotool for building Go command-line tools

    main
    The gotool package provides utility functions designed to help developers implement command-line tools that follow the semantics and behavior of the standard cmd/go tool. It is intended as a convenience library for anyone building Go-based CLI applications that require similar functionality to the official Go toolchain.
  3. Understand the Twirp Wire Protocol (v7)

    main
    Twirp is an RPC protocol based on HTTP and Protocol Buffers (proto). It uses HTTP URLs to route requests to specific RPC endpoints and transmits proto messages as the request and response bodies. The protocol supports both binary (application/protobuf) and JSON (application/json) encodings. Developers define APIs in .proto files and use Twirp tools to generate client and server libraries that implement this protocol.
  4. Construct Twirp RPC URLs

    main

    Twirp uses a specific URL structure to map requests to RPC endpoints. The format follows this ABNF pattern:

    Base-URL [ Prefix ] / [ Package. ] Service / Method

    Components:

    • Base-URL: The scheme and authority (e.g., https://example.com).
    • Prefix: An optional path prefix, commonly /twirp, but can be any arbitrary path or empty.
    • Package: The proto package name (e.g., example.calendar.v1). This is omitted if no package is defined.
    • Service: The proto service name (e.g., CalendarService).
    • Method: The proto rpc method name (e.g., CreateEvent).
  5. Understand the Twirp Wire Protocol (v5) Overview

    main

    Twirp is an RPC protocol based on HTTP and Protocol Buffers (proto). It uses HTTP URLs to specify RPC endpoints and transmits proto messages as HTTP request/response bodies.

    Key characteristics:

    • Encoding: Supports both binary (application/protobuf) and JSON (application/json) encodings.
    • Method: Always uses the HTTP POST method for requests.
    • Routing: Uses a direct mapping of URLs to service and method names.
    • Compatibility: Works with any HTTP client and any HTTP version.
  6. Understand the Twirp Wire Protocol

    main
    Twirp is a simple RPC protocol based on HTTP and Protocol Buffers (proto). It uses HTTP URLs to specify RPC endpoints and transmits proto messages as HTTP request/response bodies. The protocol supports both binary (application/protobuf) and JSON (application/json) encodings and is compatible with any HTTP client and version.
  7. How Twirp handles Protobuf and JSON serialization

    main

    Twirp's serialization handling is transparent to your service implementation.

    1. Request Handling: Twirp parses the incoming HTTP request based on the Content-Type. If the Content-Type or the body is invalid, Twirp returns an Internal error. It then converts the payload into the request struct defined in your interface.
    2. Response Handling: Your implementation returns a response struct. Twirp automatically serializes this struct back into either Protobuf or JSON, depending on the Content-Type requested by the client.
  8. Use Server Hooks to add functionality to Twirp servers

    main

    Server Hooks can be attached to a generated server constructor to provide callbacks for specific points in the request lifecycle. They are ideal for observability tasks like logging requests, recording response times, or reporting metrics.

    Key behaviors:

    • Every hook receives the request context.Context and can return a modified context.Context.
    • The Error hook is only triggered if the handler returns an error.
    • Use twirp.WithServerHooks when instantiating your server.

    Common hook callbacks include RequestRouted, Error, and ResponseSent.

    // NewLoggingServerHooks logs request and errors to stdout in the service
    func NewLoggingServerHooks() *twirp.ServerHooks {
        return &twirp.ServerHooks{
            RequestRouted: func(ctx context.Context) (context.Context, error) {
                method, _ := twirp.MethodName(ctx)
                log.Println("Method: " + method)
                return ctx, nil
            },
            Error: func(ctx context.Context, twerr twirp.Error) context.Context {
                log.Println("Error: " + string(twerr.Code()))
                return ctx
            },
            ResponseSent: func(ctx context.Context) {
                log.Println("Response Sent (error or success)")
            },
        }
    }
    
    // Usage during server instantiation:
    server := NewHaberdasherServer(svcImpl,
        twirp.WithServerHooks(NewLoggingServerHooks()))
  9. Handle Twirp Error Responses

    main

    Twirp error responses are always JSON-encoded, regardless of the original request's Content-Type. The response will always include the header Content-Type: application/json.

    An error object contains the following keys:

    • code: A string representing one of the fixed Twirp error codes.
    • msg: A human-readable string describing the error.
    • meta: (Optional) An object containing arbitrary string metadata.

    Error JSON Formats

    Basic Error:

    {
      "code": "internal",
      "msg": "Something went wrong"
    }

    Error with Metadata:

    {
      "code": "permission_denied",
      "msg": "Thou shall not pass",
      "meta": {
        "target": "Balrog",
        "power": "999"
      }
    }
  10. Twirp Protocol Spec compatibility

    main

    The Twirp Spec Protocol is the primary point of compatibility across different languages and versions.

    • V7 Spec: The current standard. Supported by Twirp Go runtime/generator versions v7.x.x and above (v8+).
    • V5 Spec: The original spec. The V7 spec is backwards compatible with V5.
    • Compatibility: Twirp Go versions labeled v5.x.x are compliant with both the V5 and V7 specs. Any service implementing the V5 spec will work with a V7 spec client.
  11. Handle default values and required fields in proto3

    main

    In proto3, all fields have zero-value defaults (e.g., "" for strings, 0 for ints) and are technically optional. Since the service implementation cannot distinguish between an empty and a missing field, use comments in the .proto file to signal intent to consumers:

    • Required Fields: Add a // required comment. This implies the server will return a twirp.RequiredArgumentError("field_name") if the field is empty.
    • Custom Defaults: Add a // (default X) comment. This implies the server will treat the zero-value as X (e.g., int32 limit = 1; // (default 20)).
    • Enums: The first item defined in the enum is the default value.
  12. Follow naming conventions for Protobuf and Twirp

    main

    Adhere to the Protocol Buffers Style Guide and Google Cloud Platform design guides:

    • Types/Messages/Services: Use CamelCase.
    • Fields: Use underscore_separated_names.
    • Enums: Use CAPITALS_WITH_UNDERSCORES for values.
    • Quantities/Durations: Include units in the field name (e.g., delay_seconds instead of delay).
    • Twitch-specific Timestamps:
      • Use names ending in _at for general timestamps (e.g., created_at, updated_at). These should be RFC3339 strings.
      • Use names ending in _time if using google.protobuf.Timestamp types.