Protobuf-ES

Protobuf-ES is a complete implementation of Protocol Buffers in TypeScript designed for modern JavaScript environments including web browsers, Node.js, Deno, and Bun. It serves as the runtime library for the protoc-gen-es code generator.

Key features include:

• Modern Ecosystem Support: Full ECMAScript module (ESM) support and first-class TypeScript support.
• Idiomatic Code: Generates idiomatic JavaScript and TypeScript code with much smaller bundle sizes compared to other implementations.
• Feature Completeness: Implements all proto3 features (including canonical JSON format) and all proto2 features (including extensions and text format).
• Standard APIs: Uses standard JavaScript APIs instead of the Closure Library.
• Advanced Capabilities: Provides descriptor and reflection support and is verified by Protocol Buffers conformance tests.

Protobuf-ES allows you to write standard Protobuf plugins using TypeScript. The @bufbuild/protoplugin package manages the plugin protocol (reading schema data from stdin and writing to stdout) and provides helpers for generating JavaScript and TypeScript code from schemas.

Plugins follow the protoc-gen-x naming convention and are implemented as executables.

Protobuf-ES provides a code size comparison against google-protobuf (via protoc-gen-js). The comparison is performed by generating code for a specific module, bundling it with esbuild, minifying the bundle, and compressing it.

As the number of generated files increases, Protobuf-ES demonstrates significantly smaller bundle sizes (both minified and compressed) compared to protobuf-javascript.

The @bufbuild/protoplugin package provides a framework for writing custom code generator plugins that are compatible with the Protobuf-ES ecosystem. These plugins can be used with both buf and protoc compilers to generate tailored TypeScript or JavaScript code from Protobuf schemas.

When building a plugin, you have three primary strategies for handling output files:

1. Full Control: Write your own generators for TypeScript, JavaScript, and declaration files.
2. Automatic JS/DTS Generation: Generate only TypeScript files and let the framework automatically produce JavaScript and declaration files using its internal TypeScript compiler.
3. Custom TypeScript Compiler: Generate only TypeScript files but provide your own TypeScript compiler to handle the generation of JavaScript and declaration files, allowing you to use specific TypeScript versions or compiler options.

Protobuf-ES is designed as a modern alternative to the official google-protobuf JavaScript generator:

Conformance for both implementations is tracked via the public Protobuf conformance runner.

Proto3 optional fields enable presence tracking for scalar and enum fields. They are generated as optional properties in TypeScript.

optional bool active = 3;

active?: boolean | undefined;

Protobuf-ES follows specific naming conventions to ensure compatibility with TypeScript and ECMAScript:

• Field Names: Uses standard lower-camel case JSON conversion (e.g., snake_case becomes snakeCase). If a name conflicts with a reserved ECMAScript keyword or a built-in property, a $ is appended to the name.
• Nested Types: Nested message and enum names are joined with underscores (e.g., User.Type becomes User_Type).
• Packages: Package declarations are mostly ignored in generated identifiers to accommodate ECMAScript's lack of Protobuf-style namespaces, though they remain visible in descriptors and fully qualified names.

Descriptors are the schema model for Protobuf. While the Protobuf compiler parses .proto files into google.protobuf.*DescriptorProto messages, Protobuf-ES wraps these in more ergonomic types. Descriptors form a hierarchy rooted at a file descriptor (DescFile).

Descriptor Hierarchy:

• DescFile (Root)
  • messages: DescMessage[]
    • fields: DescField[]
    • oneofs: DescOneof[]
    • nestedMessages: DescMessage[]
    • nestedExtensions: DescExtension[]
    • nestedEnums: DescEnum[]
  • enums: DescEnum[]
  • extensions: DescExtension[]
  • services: DescService[]

The Reflection API is composed of several key abstractions:

• Descriptors: Schema objects representing the structure of your Protobuf definitions, such as DescFile, DescMessage, DescField, DescEnum, and DescService.
• Registries: Collections of descriptors that are keyed by their fully qualified names.
• Custom options: Annotations on schema elements that are backed by extensions.
• Dynamic messages: Tools for interacting with messages at runtime, including reflect(), ReflectMessage, ReflectList, and ReflectMap.