gradle-avro-plugin

repository·master·Indexed 43 days ago

https://github.com/davidmc24/gradle-avro-plugin

A Gradle plugin for performing Java code generation from Apache Avro schemas (JSON, IDL, or Protocol). It allows configuration of code generation options such as field visibility, string types, and Optional getters, and supports registering custom logical type factories and conversions. Note: This project is end-of-life and has been donated to the Apache Avro project.

Tokens
3.3K
Snippets
12
Records
18
Agent score
56%

What's inside gradle-avro-plugin

  1. Best practices for Avro file processing

    master

    It is recommended to define each record, enum, or fixed type in its own .avsc file rather than using inline type definitions. This prevents conflicting definitions and allows the plugin to automatically resolve dependencies and compile files in the correct order.

    Example Pattern: Instead of an inline enum inside Cat.avsc, create Breed.avsc for the enum and reference it by name in Cat.avsc.

  2. Configure Optional getters for Avro fields

    master

    You can control how java.util.Optional is used in generated code using three related settings:

    1. createOptionalGetters: Creates additional methods. For a field abc, it creates Optional<T> getOptionalAbc().
    2. gettersReturnOptional: Replaces existing getters. The standard getAbc() will now return Optional<T>.
    3. optionalGettersForNullableFieldsOnly: When used with gettersReturnOptional = true, it restricts the Optional return type to only those fields defined as nullable (null unions) in the Avro schema.
    avro {
        // Returns Optional only for nullable fields
        gettersReturnOptional = true
        optionalGettersForNullableFieldsOnly = true
    }
  3. Configure a schema project to provide only .avsc files without Java classes

    master

    If you want a subproject to act strictly as a schema provider without generating Java classes, follow these steps:

    1. Relocate schemas: Rename the directory src/main/avro to src/main/resources. This ensures the .avsc files are packaged into the JAR as resources rather than being processed by the Avro plugin for code generation.
    2. Change plugin: Replace the gradle-avro-plugin with the standard Gradle java plugin in that subproject's build configuration.
  4. Install the Gradle Avro plugin

    master

    To use the plugin, add it to your Gradle build files. The plugin automatically processes an avro directory in every SourceSet (e.g., src/main/avro, src/test/avro) to generate Java classes from Avro JSON schemas, JSON protocols, or Avro IDL files.

    1. Configure settings.gradle to include the necessary repositories.
    2. Apply the plugin in build.gradle.
    3. Add an implementation dependency on Apache Avro.
    // settings.gradle
    pluginManagement {
        repositories {
            gradlePluginPortal()
            mavenCentral()
        }
    }
    
    // build.gradle
    plugins {
        id "com.github.davidmc24.gradle.plugin.avro" version "VERSION"
    }
    
    dependencies {
        implementation "org.apache.avro:avro:1.11.0"
    }
  5. Use the avro-base plugin for manual task control

    master

    If the default plugin behavior does not suit your needs, you can use the com.github.davidmc24.gradle.plugin.avro-base plugin to manually register tasks of type GenerateAvroJavaTask or GenerateAvroProtocolTask.

    import com.github.davidmc24.gradle.plugin.avro.GenerateAvroJavaTask
    
    apply plugin: "java"
    apply plugin: "com.github.davidmc24.gradle.plugin.avro-base"
    
    dependencies {
        implementation "org.apache.avro:avro:1.11.0"
    }
    
    def generateAvro = tasks.register("generateAvro", GenerateAvroJavaTask) {
        source("src/avro")
        outputDir = file("dest/avro")
    }
    
    tasks.named("compileJava").configure {
        source(generateAvro)
    }
  6. Load custom Conversions and LogicalTypeFactories from external classpath

    master

    You can load Avro Conversion and LogicalTypeFactory classes that reside outside of the standard build classpath. This requires defining a custom configuration, adding the dependency to that configuration, and then mapping the classes in the avro extension.

    configurations {
        customConversions
    }
    
    dependencies {
        customConversions(project(":custom-conversions"))
    }
    
    avro {
        conversionsAndTypeFactoriesClasspath.from(configurations.customConversions)
        logicalTypeFactoryClassNames.put("timezone", "com.github.davidmc24.gradle.plugin.avro.test.custom.TimeZoneLogicalTypeFactory")
        customConversionClassNames.add("com.github.davidmc24.gradle.plugin.avro.test.custom.TimeZoneConversion")
    }
  7. Setup the plugin using Gradle Kotlin DSL

    master

    When using the Kotlin DSL, you must apply the plugin declaratively in the plugins {} block. Configuration in the avro {} block uses property setters (e.g., .set()) rather than direct assignment.

    ```kotlin
    // In build.gradle.kts
    plugins {
        id("com.github.davidmc24.gradle.plugin.avro") version "VERSION"
    }
    
    avro {
        isCreateSetters.set(true)
        isCreateOptionalGetters.set(false)
        isGettersReturnOptional.set(false)
        isOptionalGettersForNullableFieldsOnly.set(false)
        fieldVisibility.set("PUBLIC_DEPRECATED")
        outputCharacterEncoding.set("UTF-8")
        stringType.set("String")
        templateDirectory.set(null as String?)
        isEnableDecimalLogicalType.set(true)
    }

    Note: You must also configure pluginManagement in settings.gradle.kts to include gradlePluginPortal() and mavenCentral().

  8. Generate JSON schema from Protocol (AVDL) or IDL files

    master

    To generate JSON schemas from Avro IDL (.avdl) files, you must follow a two-step process:

    1. Use GenerateAvroProtocolTask to convert .avdl to .avpr (JSON protocol).
    2. Use GenerateAvroSchemaTask to convert .avpr to .avsc (JSON schema).

    For existing .avpr files, you can use GenerateAvroSchemaTask directly.

    import com.github.davidmc24.gradle.plugin.avro.GenerateAvroProtocolTask
    import com.github.davidmc24.gradle.plugin.avro.GenerateAvroSchemaTask
    
    apply plugin: "com.github.davidmc24.gradle.plugin.avro-base"
    
    def generateProtocol = tasks.register("generateProtocol", GenerateAvroProtocolTask) {
        source file("src/main/avro")
        include("**/*.avdl")
        outputDir = file("build/generated-avro-main-avpr")
    }
    
    tasks.register("generateSchema", GenerateAvroSchemaTask) {
        dependsOn generateProtocol
        source file("src/main/avro")
        source file("build/generated-avro-main-avpr")
        include("**/*.avpr")
        outputDir = file("build/generated-main-avro-avsc")
    }
  9. Register additional Velocity tool classes

    master

    If you are overriding the default Avro Velocity templates, you can provide additional tool classes to be used during code generation. Provide a list of class names that have a default constructor. These tools will be available in your templates using the syntax $class.simpleName().

    avro {
      additionalVelocityToolClasses = ['com.yourpackage.Escape', 'com.yourpackage.JSON']
    }
  10. Configure Decimal Logical Type representation

    master

    Control how fixed or bytes fields annotated with "logicalType": "decimal" are represented in generated Java files.

    • true (default): Uses java.math.BigDecimal.
    • false: Uses java.nio.ByteBuffer.
    avro {
        enableDecimalLogicalType = false
    }
  11. Configure field visibility in generated classes

    master

    By default, fields in the generated Java files are set to PRIVATE. You can change this using the fieldVisibility option. Valid values are any org.apache.avro.compiler.specific.SpecificCompiler.FieldVisibility constant or an equivalent case-insensitive string (e.g., "PUBLIC").

    avro {
        fieldVisibility = "PUBLIC"
    }