DOCSEARCHbyAlgolia
Submit Docs

Apicurio Registry Documentation

repository·main·Indexed 21 days ago

https://github.com/apicurio/apicurio-registry

An API and Schema registry for storing and retrieving APIs and Schemas via REST API and web UI. Features include a Confluent Schema Registry compatibility layer (v7 and v8) for AVRO, JSON Schema, and Protobuf, HTTP caching support via reverse proxies (e.g., Varnish, Nginx), and comprehensive health and performance monitoring for storage and REST components.

Tokens
245.9K
Snippets
602
Records
1K
Agent score
69%

What's inside Apicurio Registry

  1. Overview of the ODCS Data Contracts Demo Flow

    main

    The demo covers the lifecycle of data contracts through 8 phases, including schema registration, contract submission, quality scoring, CEL rule verification, schema migration, and Kafka SerDes integration.

    Key capabilities demonstrated include:

    • Registering Avro schemas with inline PII tags.
    • Submitting ODCS contracts for projection (rules, labels, tags).
    • Promoting contracts from DEV to STAGE based on quality scores.
    • Verifying projected CEL (Common Expression Language) rules.
    • Managing schema migrations (e.g., JSONata migration).
    • Auditing and searching contracts.
    • Setting global organization-wide contract rules.
    • Kafka integration: producing valid/invalid messages and consuming them via SerDes.

  3. Overview of Apicurio Registry Kafka Examples

    main

    The repository contains several patterns for integrating Apicurio Registry with Kafka applications:

    • Simple Avro/JSON Schema: Basic publish/subscribe using Avro for serialization or JSON Schema for payload validation.
    • Confluent Serdes Integration: Using Apicurio Registry deserializers alongside Confluent serializers in mixed environments.
    • Avro Bean Example: Using Java Beans as Kafka message payloads instead of GenericRecord.
    • Custom ID Strategy: Implementing a custom Global ID strategy for schema lookups by producers.
    • Maven Plugin Integration: Using the Apicurio Registry Maven plugin to pre-register schemas during the build process.
    • Mixed Avro Schemas: Handling multiple different Avro schemas within the same Kafka topic.
    • Real-world Order Processing: A complete Quarkus-based architecture simulating an order system with automatic schema registration and evolution.

  4. What is Apicurio Registry?

    main

    Apicurio Registry is a datastore for sharing event schemas and API designs across event-driven and API architectures. It allows you to decouple data structures from client applications by providing a REST interface to manage and query schemas at runtime.

    Key features include:

    • Support for multiple formats (Avro, JSON Schema, Protobuf, AsyncAPI, OpenAPI, etc.).
    • AI/ML artifact support (MODEL_SCHEMA and PROMPT_TEMPLATE).
    • Pluggable storage (PostgreSQL, Apache Kafka, or embedded H2).
    • Content validation and compatibility rules.
    • Full Apache Kafka schema registry support and Confluent client compatibility.

  5. Overview of Apicurio Registry GitOps Sync

    main

    Apicurio Registry GitOps Sync is a lightweight synchronization container (quay.io/apicurio/apicurio-registry-gitops-sync) designed for the Apicurio Registry's GitOps storage mode. It manages a local Git repository on a shared volume that the registry reads from, enabling the registry to use Git as its source of truth.

    Key Modes

    • Pull mode (default): Periodically fetches from a remote Git repository to keep the local clone up to date. The registry automatically detects and reloads changes.
    • Push mode: Runs an SSH server that accepts git push from CI/CD pipelines or developers. Updates become available to the registry on the next poll cycle.
    • Mixed mode: In multi-repo setups, individual repositories can be configured for either pull or push mode independently.
    quay.io/apicurio/apicurio-registry-gitops-sync

  6. Overview of XSD Compatibility Checker

    main

    The XSD Compatibility Checker is a component of the Apicurio Registry that validates XML Schema Definition (XSD) evolution. It ensures that changes to schemas do not break existing data producers or consumers by enforcing industry-standard compatibility rules.

    Supported Compatibility Modes

    • BACKWARD compatibility: Ensures old data is readable by the new schema. This is the most common mode used when you want new schema versions to accept data created with older versions.
    • FORWARD compatibility: Ensures new data is readable by the old schema. This is used when you want older schema versions to remain functional even after newer data is produced.
    • FULL compatibility: The strictest mode; requires both backward and forward compatibility to be satisfied.
    • TRANSITIVE modes: Extends compatibility checks across the entire version history. Instead of just checking the immediate predecessor, it validates the new schema against every historical version in the registry.

  8. Available Apicurio Registry SDKs

    main

    You can manage artifacts in the Apicurio Registry (create, read, update, delete) and perform administrative functions (managing global rules, importing/exporting data) using one of the following language-specific SDKs:

    • Java: Accessed via Apache Maven dependency. Uses the JDK HTTP client and can be customized for TLS or custom headers.
    • TypeScript: Accessed via package.json (for Node.js applications).
    • Python: Accessed via PyPI.
    • Golang: Accessed via Go dependency management.

  9. Run Apicurio Registry in GitOps mode

    main

    In GitOps mode, the Apicurio Registry is read-only. All schemas, artifacts, groups, and rules are managed by modifying files in a Git repository. The registry loads data from the repository, and changes are reflected after the registry or its sidecar detects new commits.

    Deployment Modes

    ModeDescriptionSidecar RequiredSecurity Level
    Local VolumeClones a Git repo locally and mounts it into the registry container. Best for experimentation.NoN/A
    Pull HTTPSUses a sidecar to pull from a public Git repository over HTTPS.Yesdev
    Pull SSHUses a sidecar to pull from a private Git repository via SSH using a deploy key.Yesdev
    Multi-Repo HTTPSAggregates schemas from multiple Git branches/repos into one registry.Yesdev
    PushThe sidecar runs an SSH server that accepts git push directly. Useful for restricted networks or CI/CD.Yesdev

    Endpoints

    • Registry API: http://localhost:8080/apis/registry/v3
    • Registry UI: http://localhost:8888

  10. Interact with Apicurio Registry via various interfaces

    main

    Users can interact with the registry through several interfaces depending on their needs:

    • Web UI: A React-based application (built with PatternFly) for browsing, searching, and managing artifacts, versions, groups, and rules.
    • REST API: Full CRUD operations via the Registry v3 REST API. The OpenAPI spec is available at /apis/registry/v3.
    • Client SDKs: Native SDKs for Java, Go, and TypeScript (generated via Kiota).
    • CLI: Command-line interface for automation and scripting.
    • Serializer/Deserializer Libraries: Integration into Kafka, Pulsar, and NATS clients for transparent schema resolution.
    • Maven Plugin: For schema registration and downloading during the build process.
    • Kubernetes Operator: For declarative deployment and lifecycle management.
    • MCP Server: Allows AI/LLM tools to interact with the registry via the Model Context Protocol.

  12. Configure Apicurio Registry storage options

    main

    Apicurio Registry supports several storage backends to persist schema data. Depending on your deployment environment and security requirements, you can choose from the following storage types:

    • In-memory storage: Useful for testing or ephemeral environments (available in specific builds).
    • SQL-based storage: Uses a relational database.
    • KafkaSQL-based storage: Uses Kafka as the underlying storage mechanism, with various security configurations:
      • plain: No security/encryption.
      • tls: Uses TLS for encryption.
      • scram: Uses SCRAM authentication.
      • oauth: Uses OAuth2 authentication.
      • access: Uses access control mechanisms.

    If you are using the Apicurio Operator on OpenShift, these configurations are typically managed via the OperatorHub UI. For Kubernetes deployments, the same configuration options are available via command-line tools (e.g., kubectl).