kotlinx-rpc

repository·main·Indexed 22 days ago

https://github.com/kotlin/kotlinx-rpc

A Kotlin library for adding asynchronous Remote Procedure Call (RPC) services to applications using suspend functions and Flows. It includes a compiler plugin for generating stub classes and descriptors via the @Rpc annotation, a Gradle plugin (org.jetbrains.kotlinx.rpc.plugin) for Protobuf/gRPC code generation using the buf CLI, and protocol-specific implementations for kRPC and gRPC (supporting JVM and Native).

Tokens
17.5K
Snippets
55
Records
102
Agent score
77%

What's inside kotlinx-rpc

  1. Overview of the kotlinx-rpc Protobuf runtime

    main
    The Protobuf runtime is a multiplatform implementation designed to be fully independent of the rest of the library and external code generation. It utilizes platform-specific protobuf libraries to handle wire encoding and decoding. It is primarily intended to be used alongside code generated by protoc-gen plugins found in the /protoc-gen directory.
  2. Overview of kotlinx-rpc protoc plugins

    main

    The protoc-gen package provides two code generators that translate .proto files into idiomatic Kotlin multiplatform code. These plugins are designed to be invoked by Google's protoc compiler or Buf's protoc compiler using the standard plugin protocol (receiving a binary CodeGeneratorRequest on stdin and emitting a CodeGeneratorResponse on stdout).

    Key Integration Details:

    • Distribution: The plugins are distributed as fat JARs.
    • Consumption: They are consumed by the kotlinx-rpc Gradle plugin during user builds.
    • Dependencies: The code generated by these plugins depends on the protobuf/ runtime modules.
    • Platform: This specific build is JVM-only.
  3. Use gRPC protocol support in kotlinx-rpc

    main

    The grpc module provides a Kotlin Multiplatform client/server gRPC implementation. It is designed to work across different targets by leveraging specific underlying implementations:

    • JVM: Uses gRPC-Java.
    • Native: Uses C interop with the gRPC Core library.

    Note on platform support:

    • Supported: JVM, Native.
    • Not yet supported: WASM, JS, and MinGW.
  4. Understand the gradle-conventions build logic

    main

    The gradle-conventions module is an included build (not a standalone project) that provides precompiled Gradle convention plugins for kotlinx-rpc. It standardizes how subprojects are built, tested, published, and documented.

    It works alongside gradle-conventions-settings/, which provides settings-phase plugins for repositories, version resolution, and Develocity. Both are included via the root settings.gradle.kts.

  5. Project Structure of the gRPC KMP Sample

    main

    This sample project demonstrates kotlinx.rpc for gRPC using a Kotlin Multiplatform (KMP) architecture. The project is organized as follows:

    • composeApp/src: Contains code shared across Compose Multiplatform applications. commonMain holds the gRPC client application code used by all platforms.
    • server/src/main/kotlin: Contains the gRPC server application implementation.
    • shared/src: Contains the shared code used by both the server and client, including .proto files and the generated code.
  6. Understand the project structure of the Ktor All-Platforms Sample

    main

    This sample project is a Kotlin Multiplatform (KMP) application targeting Android, iOS, Desktop, and Server. It demonstrates how to use kotlinx-rpc across different platforms. The directory structure is organized as follows:

    • /composeApp: Contains code shared across Compose Multiplatform applications.
      • commonMain: Code common to all targets.
      • Platform-specific folders (e.g., iosMain): Code compiled only for a specific platform (e.g., for calling Apple's CoreCrypto on iOS).
    • /iosApp: The entry point for the iOS application. This is where you add SwiftUI code.
    • /server: The Ktor server application.
    • /shared: Code shared between all targets. The most important subfolder is commonMain.
  7. Understand the native-deps structure and build flow

    main

    The native-deps directory provides standalone builds and shared tooling for native dependency artifacts used by Kotlin/Native components of kotlinx-rpc.

    Component Structure

    • grpc-c-prebuilt/: Builds and publishes raw native gRPC bundles and public headers (contains .a static libraries in .zip artifacts, no Kotlin code).
    • shims/: Contains Kotlin/Native shim builds, including gRPC, protobuf, and shared shim annotations.
    • bazel-support/: Shared Bazel and Kotlin/Native toolchain support.

    The Shim Model

    Shims are used to bridge native libraries to Kotlin/Native via cinterop. The process follows this flow:

    1. grpc-c-prebuilt/ publishes prebuilt grpc-c core static archives (.a) and public headers.
    2. A shim module consumes these archives, produces cinterop KLIBs, and patches its internal cinterop package.
    3. The patched KLIB is published alongside its annotation artifact.
    4. Fixture tests verify artifact contents and downstream compiler behavior.

    Key Shim Modules

    • shims/grpc/: Publishes the gRPC shim artifact and runs cinterop.
    • shims/protobuf/: Publishes the protobuf shim artifact and runs cinterop.
    • shims/annotation/: Publishes shared opt-in markers required by shim consumers.
    • shims/klib-patcher/: Internal build tooling only. It patches cinterop KLIB metadata to ensure shim declarations require explicit opt-in. It is not part of the public RPC compiler plugin story.
  8. What is kRPC

    main
    kRPC is a custom bidirectional RPC protocol provided by the kotlinx-rpc library. It is designed for message-based communication and uses pluggable transports, with Ktor WebSockets serving as the canonical transport implementation. It is one of two available protocol implementations in the library, the other being gRPC.
  9. Manage gRPC versioning and Bazel synchronization

    main

    Versioning and configuration for the prebuilt gRPC artifacts follow these rules:

    • Source of Truth: The gRPC version is defined in versions-root/libs.versions.toml via internal-native-grpc-shim.
    • Bazel Synchronization: Gradle automatically rewrites MODULE.bazel before Bazel-backed build, package, or publish tasks to ensure the Bazel grpc dependency remains in sync with the project version.
    • Warning: Do not manually edit GRPC_VERSION in MODULE.bazel, as these changes will be overwritten during the next Gradle run.
  10. Understand the Protobuf module structure

    main

    The Protobuf implementation is organized into three main modules:

    1. protobuf-lite: Contains the core wire encoders/decoders, the base message class, the descriptor system, extensions, and configuration.
    2. protobuf-wkt: Provides Google Well-Known Types (e.g., Any, Duration, Timestamp, Struct, Descriptor).
    3. protobuf: An aggregator module that re-exports protobuf-lite, protobuf-wkt, and grpc-marshaller for easier consumption.
  11. Understand the gRPC module dependency structure

    main

    The gRPC implementation is composed of several layers that build upon the core RPC abstractions:

    • grpc-ktor-server depends on grpc-server.
    • grpc-server and grpc-client both depend on grpc-core.
    • grpc-core depends on grpc-marshaller.
    • grpc-marshaller depends on :core (the base RPC abstractions).
    • grpc-marshaller-kotlinx-serialization provides serialization support for the marshaller.

    For details on the code generated from protobuf definitions, refer to the protoc-gen codegen documentation.

  12. Understand kRPC compatibility testing

    main

    The repository contains two specialized test suites to ensure protocol and API stability:

    1. krpc-compatibility-tests: Uses separate ClassLoaders to load different API versions, verifying that clients and servers can communicate across different versions of the API.
    2. krpc-protocol-compatibility-tests: Uses template-generated code for each protocol version to perform matrix testing across all version pairs using dynamic JUnit 5 test factories.