Pbandk: Kotlin Multiplatform Protocol Buffers

repository·master·Indexed 43 days ago

https://github.com/streem/pbandk

A Kotlin Multiplatform (KMP) code generator and runtime for Protocol Buffers that generates idiomatic Kotlin data classes from .proto files. It supports proto2 and proto3 syntaxes, JSON serialization, and targets JVM, Android, iOS, macOS, Linux, Windows, JS, and Wasm. Pbandk uses protoc for code generation and provides a runtime library available on Maven Central.

Tokens
5.7K
Snippets
23
Records
32
Agent score
55%

What's inside Pbandk

  1. What is Pbandk?

    master
    Pbandk is a Kotlin Multiplatform (KMP) code generator and runtime for Protocol Buffers. It is designed to generate idiomatic Kotlin code from .proto definitions, supporting all platforms compatible with Kotlin Multiplatform, including JVM, Android, iOS, macOS, Linux, Windows, JS, and Wasm.
  2. Platform support and limitations in Pbandk

    master

    Pbandk is a Kotlin Multiplatform library, but support levels vary by platform:

    Stable/Primary Platforms

    • JVM, Android, and Native (iOS, macOS, Linux): Fully supported.
    • Windows (mingwX64): Supported on a best-effort basis. While the runtime is available, full correctness cannot be guaranteed via official protobuf conformance tests on Windows, though it shares most code with Linux/macOS.

    Experimental Platforms

    • Kotlin/JS (js): Support is experimental. While using pbandk from other KMP code on JS is mostly stable, using it directly from Javascript/Typescript is not well supported due to @JsExport limitations.
    • Kotlin/Wasm (wasmJs): Support is experimental. Pbandk APIs and generated code can only be called from other Kotlin code compiled for Wasm; they cannot be called directly from Javascript or other Wasm host languages due to limitations in passing complex types to the Wasm host environment.
  3. Understand the structure of generated Pbandk code

    master

    Pbandk generates Kotlin data classes that implement the pbandk.Message interface. These classes represent Protobuf messages and include several key features:

    • Data Classes: Messages are represented as standard Kotlin data classes, making them easy to use with standard Kotlin features.
    • Companion Objects: Each message has a Companion object that provides a defaultInstance and a decodeWith(u: pbandk.MessageDecoder) method for deserialization.
    • Enums: Protobuf enums are generated as sealed classes implementing pbandk.Message.Enum. They include a values list and helper methods like fromValue(value: Int) and fromName(name: String).
    • Unknown Fields: Every message includes an unknownFields property of type Map<Int, pbandk.UnknownField> to preserve data that was not recognized during decoding.
    • Merging: Messages implement the plus operator (operator fun plus(other: pbandk.Message?)) to allow merging two messages together.
    • Descriptors: Each message provides a descriptor property of type pbandk.MessageDescriptor for runtime reflection and schema inspection.
    • Default Values: The library provides orDefault() extension functions (e.g., Person?.orDefault()) to safely handle nullable message instances by returning the defaultInstance if the receiver is null.
    // Example of a generated message structure
    @pbandk.Export
    public data class Person(
        val name: String = "",
        val id: Int = 0,
        val email: String = "",
        val phones: List<tutorial.Person.PhoneNumber> = emptyList(),
        val lastUpdated: pbandk.wkt.Timestamp? = null,
        override val unknownFields: Map<Int, pbandk.UnknownField> = emptyMap()
    ) : pbandk.Message {
        override operator fun plus(other: pbandk.Message?): tutorial.Person = protoMergeImpl(other)
        override val descriptor: pbandk.MessageDescriptor<tutorial.Person> get() = Companion.descriptor
        // ...
        public companion object : pbandk.Message.Companion<tutorial.Person> {
            public val defaultInstance: tutorial.Person by lazy { tutorial.Person() }
            override fun decodeWith(u: pbandk.MessageDecoder): tutorial.Person = Person.decodeWithImpl(u)
            // ...
        }
    }
  4. How generated Pbandk messages work

    master

    Generated messages are immutable Kotlin data classes that extend pbandk.Message.

    Serialization and Deserialization

    • Binary: Use encodeToByteArray() on the instance and MessageName.Companion.decodeFromByteArray(byteArray) to deserialize.
    • JSON: Use encodeToJsonString() and decodeFromJsonString() for Protobuf JSON encoding.
    • Streams (JVM only): Use encodeToStream(outputStream) and decodeFromStream(inputStream).

    Key Features

    • Immutability: Messages are data classes, providing equals, hashCode, and toString automatically.
    • Unknown Fields: An unknownFields map stores unrecognized fields. These are preserved and re-encoded during output. You can configure MessageDecoder to discard them if desired.
    • Merge Semantics: The plus operator is defined for messages to follow protobuf merge semantics.
    • Size: The protoSize field lazily calculates the message size.
    • Bytes: bytes fields use a pbandk.ByteArr wrapper around ByteArray to ensure predictable behavior in Kotlin data classes.
  5. How Repeated and Map fields are represented

    master

    Pbandk maps Protobuf collection types to standard Kotlin collections:

    • repeated fields: Represented as standard Kotlin Lists. Do not assume a specific implementation (like ArrayList).
    • map fields: Represented as standard Kotlin Maps.

    Note for proto2: In proto2, because of how map entries are serialized, both the key and the value are considered nullable.

  6. How Enums are represented in Pbandk

    master

    Because Protobuf enums are open-ended (they can contain values not defined in the schema), Pbandk generates them as sealed classes rather than standard Kotlin enum classes.

    Structure

    • Each known enum value is a nested object.
    • An Unrecognized nested class is provided to hold unknown values encountered during decoding.
    • Each enum object contains a value (numeric) and a name (string).

    Usage

    • Mapping: Use the companion object's fromValue(int) or fromName(String) methods to map values to the sealed class instances.
    • Iteration: Use the companion object's values field to get a list of all known enum values.
    • Exhaustive Checks: When using when expressions, you must explicitly handle the Unrecognized case.
  7. Key features of Pbandk

    master

    Pbandk provides several features for working with Protocol Buffers in Kotlin:

    • Idiomatic Code Generation: Generates clean Kotlin data classes.
    • Syntax Support: Supports both proto2 and proto3 syntaxes.
    • Oneof Handling: oneof fields are generated as Kotlin sealed classes.
    • Well-Known Types: Specialized support for wrappers (e.g., StringValue, BoolValue) which are mapped to nullable primitives (e.g., String?, Boolean?).
    • JSON Support: Implements JSON serialization/deserialization following the proto3 JSON spec.
    • Extensibility: Supports custom service/gRPC code generation and custom protobuf options.
    • Deprecation: Respects the deprecated protobuf option in generated code.
  8. Project Structure of the Browser JS Example

    master

    The Browser JS example is organized into two sub-projects to manage dependencies correctly:

    1. :lib-proto: This project runs the pbandk protobuf plugin on the .proto files. The generated code is added to the commonMain source set.
    2. :app: This is the main Kotlin/JS project that consumes the generated code.

    This separation is necessary because the Protobuf Gradle Plugin requires the java-library plugin, while the :app project is kept as a standard Kotlin Multiplatform project for easier maintenance.

  9. How Well-Known Types (WKT) are handled

    master

    Types imported from google/protobuf (Well-Known Types) are mapped to the pbandk.wkt package.

    Specialized Mapping

    Pbandk provides specialized support for google.protobuf/wrappers.proto to map them into Kotlin nullable primitives. For example:

    • google.protobuf.StringValue $\rightarrow$ String?
    • google.protobuf.Int32Value $\rightarrow$ Int?
  10. How Oneof and Enum fields are represented

    master

    Pbandk uses Kotlin-idiomatic types to represent Protobuf's open-ended structures.

    Oneof Fields

    oneof fields are generated as nested classes within a common sealed base class.

    • Each field in the oneof is a class wrapping a single value.
    • The parent message contains nullable fields for every oneof member. Accessing a field returns its value if that specific oneof case is set, otherwise it returns null.
  11. Use pbandk in the browser via Kotlin/JS

    master

    The browser-js example shows how to leverage pbandk in web environments using Kotlin's JavaScript integration. This is useful for projects targeting the JS runtime where you need to handle Protobuf data.

    See the `examples/browser-js` directory for implementation details.
  12. Write a custom code generator for services

    master

    The custom-service-gen example provides a pattern for extending pbandk by writing a custom code generator specifically for service definitions. This allows you to generate specialized logic or boilerplate beyond the standard message types.

    See the `examples/custom-service-gen` directory for implementation details.