Eclipse EDC Connector Documentation

repository·main·Indexed 19 days ago

https://github.com/eclipse-edc/connector

A framework for managing secure data transfers and interactions within a data space. The documentation covers the connector-core foundational layer, the Catalog Crawler module for Federated Catalog Cache, transfer state machine configuration, and developer guides for integration testing using Docker-compose or ClassLoaders. It also includes details on observability via Micrometer metrics, Prometheus scraping, OpenTelemetry distributed tracing, and deployment using Helm charts for Kubernetes.

Tokens
79.9K
Snippets
179
Records
345
Agent score
64%

What's inside EDC Connector

  1. Overview of the Catalog Crawler module

    main

    The Catalog Crawler module provides implementations for the Federated Catalog Cache. This cache is a database that maintains a snapshot of all catalogs offered by all connectors within a dataspace.

    To populate and maintain this snapshot, the module utilizes Crawler objects that execute periodically to collect catalog data.

    Note: This module is currently under development and is not considered stable.

  2. Overview of Connector-core

    main
    The connector-core extension serves as the foundational layer for the EDC Connector. It is responsible for registering default and core services, including essential implementations such as the HTTP client and in-memory stores. Developers use this extension to ensure that the basic infrastructure required for the connector to function is available by default.
  3. JsonWebSignature2020 Crypto Suite Overview

    main

    The JsonWebSignature2020 module provides an implementation for the JsonWebSignature2020 crypto suite. It acts as glue code between the iron-verifiable-credentials library and the Nimbus JOSE library.

    Key Components:

    • Vc class: The primary entrypoint for signing/issuing and verifying JSON-LD structures.
    • JwsSignature2020Suite: The suite configuration used for cryptographic operations.
    • JsonWebKeyPair: Represents the keys used for signing.

    Important Requirements:

    • Familiarity with JSON-LD is required as VerifiableCredentials and VerifiablePresentations are processed as JSON-LD objects.
    • Curve Support: Because Java 17 dropped support for the secp256k1 curve, this implementation does not include it to maintain a smaller footprint. Use other available curves.
    • Octet Keypairs (OKP): Support for OKP requires an additional tink dependency. If you do not want this dependency, you must exclude it in your build script.
  4. Observability with Micrometer metrics

    main

    The EDC uses the Micrometer metrics library to automatically collect metrics from the host system, JVM, and various frameworks used within the connector (such as OkHttp, Jetty, and Jersey).

    Key characteristics of the metrics implementation:

    • Optional Instrumentation: Metrics collection is optional and can be disabled globally or for specific modules via runtime settings.
    • Decoupled Dependencies: Micrometer dependencies are not part of the core module. Instead, middleware and callbacks are used to optionally import monitoring modules at runtime.
    • Integration: By using the global Micrometer registry, the EDC can integrate with observability stacks like Prometheus or Azure Application Insights (via the Application Insights agent).
  5. Configure SQL Pool with Apache Commons Pool

    main

    The sql-pool-apache-commons extension allows you to register named javax.sql.DataSource instances to the org.eclipse.edc.transaction.datasource.spi.DataSourceRegistry. This enables the pooling of java.sql.Connections using the Apache Commons Pool library.

    Since version 0.3.1, a new configuration schema must be used. Ensure you use the edc.datasource.<datasource_name>.pool.connections.* and edc.datasource.<datasource_name>.pool.connection.test.* keys to avoid using deprecated settings.

  6. Understand the Decentralized Identity (DID) extension modules

    main

    The decentralized-identity extension provides modules for implementing Decentralized Identifier (DID) use cases. The extension is composed of the following modules:

    • identity-did-spi: Extension points for the distributed identity subsystem.
    • identity-did-crypto: Cryptographic utilities.
    • identity-did-core: Core services, including the DidResolverRegistry.
    • identity-did-service: Implementation of the IdentityService interface via DecentralizedIdentifierService.
    • identity-did-web: Support for resolving Web DIDs.
    • registration-service: A periodic job that crawls the ION network for DIDs of a specific type.
    • registration-service-api: REST API for the registration-service.
    • identity-did-test: Utility for loading RSA keys from a *.jks file.

    WARNING: These modules are under development and should not be used in production. APIs, interfaces, and implementations are subject to change without notice.

  7. Components of the SQL Common library

    main

    The SQL Common library provides abstractions for interacting with relational databases via JDBC. Its core components include:

    • SqlQueryExecutor: Executes SQL queries against a java.sql.Connection.
    • ConnectionPool: An interface for managing multiple java.sql.Connection instances.
    • ConnectionFactory: An interface for creating java.sql.Connection instances for specific database types (e.g., PostgreSQL).
    • ConnectionFactoryDataSource: A utility to construct a java.sql.DataSource from a ConnectionFactory for registration in the org.eclipse.edc.transaction.datasource.spi.DataSourceRegistry.
    • ConnectionPoolDataSource: A utility to construct a java.sql.DataSource from a ConnectionPool for registration in the org.eclipse.edc.transaction.datasource.spi.DataSourceRegistry.
  8. SQL-based TransferProcessStore Overview

    main

    The transfer-process-store-sql extension provides a SQL-based implementation of the TransferProcessStore interface. This implementation is specifically designed and tested for PostgreSQL. While it may work with other RDBMS, using other databases may result in unexpected side effects.

    Key architectural details:

    • Schema Definition: The table schema is defined in src/main/resources/transfer-process-schema.sql.
    • Statement Management: SQL constants (table names, column names, and statement templates) are managed via the TransferProcessStoreStatements interface. While column names are standardized, specific SQL implementations are handled in the PostgreStatements class.
  9. Use the SQL Policy Store for policy persistence

    main

    The SQL Policy Store provides a mechanism for persisting policy definitions using a relational database.

    Important Compatibility Note: The SQL statements (DDL) provided with this extension are specifically designed and tested for PostgreSQL. While it may function with other RDBMS, using a different database may lead to unexpected side effects.