repository·main·Indexed 23 days ago
https://github.com/capnproto/go-capnpGo language support for Cap'n Proto, including standard schemas and the v3 package. It provides tools for generating Go code from schemas using the capnp compiler, as well as core types for memory addressing (Address, Size, DataOffset), asynchronous RPC result handling (Answer, Future), and promise pipelining via PipelineCaller and AnswerQueue.
Compiling a schema produces a .capnp.go file containing Go structs that represent your schema definitions. These structs provide accessor and mutator methods for the fields defined in the schema.
For a field defined as title @0 :Text; in Cap'n Proto, the generated Go code will include:
Title() (string, error)SetTitle(string) errorCap'n Proto implements streaming without a dedicated streaming construct. Instead, you define an interface that mimics a stream (e.g., a write method for data chunks and a done method to signal completion).
To simplify the Go implementation, use the stream return type in your Cap'n Proto schema. When a method's return type is marked as stream, the generated Go method returns only an error instead of a Future and ReleaseFunc. This error will be non-nil if any prior streaming call in the sequence failed, allowing for easy short-circuiting.
interface ByteStream {
write @0 (data :Data) -> stream;
done @1 ();
}Cap'n Proto RPC is a distributed object protocol based on an object capability model. Instead of addressing global URLs or singleton objects, you interact with remote objects via capabilities (references to remote objects).
Capabilities are first-class objects, meaning they can be:
struct.List type.This allows for dynamic object creation at runtime and enables object-oriented programming (OOP) patterns over the network.
To successfully generate Go code, capnpc-go requires two specific $Go annotations in your .capnp schema file. Without these, compilation will fail.
$Go.package("package_name"): Specifies the Go package name that will appear at the top of the generated .go files.$Go.import("import_path"): Declares the full import path within your project. This is used by the compiler to generate correct import statements when schemas import types from other schemas.Example schema structure (foo/books.capnp):
using Go = import "/go.capnp";
@0x85d3acc39d94e0f8;
$Go.package("books");
$Go.import("foo/books");
struct Book {
title @0 :Text;
pageCount @1 :Int32;
}using Go = import "/go.capnp";
@0x85d3acc39d94e0f8;
$Go.package("books");
$Go.import("foo/books");
struct Book {
title @0 :Text;
# Title of the book.
pageCount @1 :Int32;
# Number of pages in the book.
}Instantiating a type generated from a schema follows a three-step hierarchy. You do not interact with the raw buffer directly; instead, you wrap it in layers of abstraction:
capnp.Arena: A low-level wrapper around a []byte buffer. Use capnp.SingleSegment(nil) for a standard new arena.*capnp.Message: Allocates Cap'n Proto structs within an arena. Use capnp.NewMessage(arena) to create a new message and obtain its root segment.Message that provides getter and setter methods. Use the NewRootXXX function (e.g., books.NewRootBook(seg)) to instantiate the top-level struct of a message.Each field in your schema produces a getter (e.g., Title()) and a setter (e.g., SetTitle()).
// 1. Create Arena
arena := capnp.SingleSegment(nil)
// 2. Create Message
msg, seg, err := capnp.NewMessage(arena)
if err != nil {
panic(err)
}
// 3. Create Schema Type (Root)
book, err := books.NewRootBook(seg)
if err != nil {
panic(err)
}
// Use Getters/Setters
_ = book.SetTitle("War and Peace")
title, _ := book.Title()The standard schemas are organized to facilitate easy importing and package separation:
/std/capnp/${schema_name}.capnp./std/capnp/${mangled_schema_name}/${mangled_schema_name}.capnp.go./std/go.capnp contains annotations used by go-capnpc. Its generated source is placed in the root of the repository, making it part of the capnproto.org/go/capnp/v3 Go package.To generate Go code from Cap'n Proto schemas, you must install the capnpc-go compiler plugin using go install. This installs the capnpc-go executable into your $(go env GOPATH)/bin directory. Ensure that this directory is included in your shell's $PATH variable so the capnp tool can locate the plugin during compilation.
go install capnproto.org/go/capnp/v3/capnpc-go@latestTo use the standard schemas provided in this repository within your own Cap'n Proto schemas, you must include the directory containing the std folder in your include path (-I). This allows $import statements in your schemas to resolve correctly to the base schemas.
Use the following command structure to compile your schema:
capnp compile -I ${path_to_this_repository}/std -ogo ${schema_name}.capnp
capnp tool with the Go plugin, then build and run the Go server. This example demonstrates RPC capabilities using hashes.Installation.md guide for specific instructions on setting up the necessary compilers and environment.When calling write() in a loop, Cap'n Proto does not provide backpressure by default, which can lead to high memory usage and latency. To prevent this, attach a flow limiter from the flowcontrol package to your capability using SetFlowLimiter. This will cause future RPC calls to block if the amount of in-flight data exceeds the specified limit.
Use flowcontrol.NewFixedLimiter(n) to limit in-flight data to n bytes.
import "capnproto.org/go/capnp/v3/flowcontrol"
// ...
// Limits in-flight data to 2^16 bytes = 64KiB:
client.SetFlowLimiter(flowcontrol.NewFixedLimiter(1 << 16))To stream multiple objects over a connection (like a network socket or file) rather than sending a single discrete buffer, use the Encoder and Decoder types. These handle the 'framing' required to separate objects in a continuous byte stream.
capnp.NewEncoder(io.Writer) (or capnp.NewPackedEncoder for compressed data).encoder.Encode(msg) where msg is the *capnp.Message (you can get this via yourStruct.Message()).capnp.NewDecoder(io.Reader) (or capnp.NewPackedDecoder for compressed data).decoder.Decode() to get a *capnp.Message.ReadRootXXX function to access the data.