CloudEvents Python SDK

repository·main·Indexed 43 days ago

https://github.com/cloudevents/sdk-python

A Python SDK for creating, parsing, and transmitting CloudEvents. It supports structured and binary formats over HTTP, Kafka, and AMQP 1.0. The SDK includes a v2 core API featuring a protocol-based design, immutable events, and strict validation, while providing a cloudevents.v1 compatibility layer for legacy migrations.

Tokens
12.7K
Snippets
37
Records
57
Agent score
55%

What's inside CloudEvents

  1. Implement a custom Serialization Format in v2

    main

    v2 uses the Format protocol instead of marshaller callbacks. To support a new format (like YAML), implement the Format class with the following methods:

    • read(event_factory, data): Parse data into attributes and return a BaseCloudEvent.
    • write(event): Serialize the entire event to bytes.
    • write_data(data, datacontenttype): Serialize just the data payload.
    • read_data(body, datacontenttype): Deserialize just the data payload.
    • get_content_type(): Return the MIME type.

    Example usage:

    from cloudevents.core.bindings.http import to_binary
    
    # Use your custom format implementation
    message = to_binary(event, event_format=YAMLFormat())
  2. Quickstart: Send CloudEvents via HTTP JSON

    main

    This sample demonstrates how to run a local server and a client to exchange structured and binary CloudEvents over HTTP using JSON.

    1. Install dependencies: Ensure you have the necessary packages installed using the provided requirements file.
    2. Start the server: Run the server script to listen for incoming CloudEvents.
    3. Run the client: Execute the client script, passing the server's URL as an argument, to send both structured and binary CloudEvents to the server.
    # 1. Install dependencies
    pip install -r requirements.txt
    
    # 2. Start the server
    python json_sample_server.py
    
    # 3. In a new shell, run the client
    python client.py http://localhost:3000/
  3. Quickstart: Send Image Payloads via HTTP

    main

    This guide demonstrates how to run a local server and a client to send CloudEvents containing image payloads using both structured and binary formats.

    1. Setup the Server

    First, install the necessary dependencies and start the sample server:

    pip install -r requirements.txt
    python image_sample_server.py

    2. Run the Client

    In a separate terminal, execute the client script to send a CloudEvent (containing an image) to the running server at http://localhost:3000/:

    python client.py http://localhost:3000/
    # Install dependencies
    pip install -r requirements.txt
    
    # Start the server
    python image_sample_server.py
    
    # Run the client (in a new shell)
    python client.py http://localhost:3000/
  4. Migrate from CloudEvents SDK v1 to v2

    main

    This guide outlines the breaking changes and new patterns introduced in v2 of the CloudEvents Python SDK.

    Requirements

    • Python: v2 requires 3.10+ (v1 supported 3.7+).
    • Dependencies: v2 requires python-dateutil>=2.8.2 only. The optional pydantic extra from v1 is no longer required.

    Architectural Shifts in v2

    1. Protocol-based design: BaseCloudEvent is now a Protocol, not a base class. Events use explicit getter methods instead of dict-like access.
    2. Explicit serialization: Implicit JSON handling is replaced by a Format protocol (e.g., JSONFormat).
    3. Strict validation: Events are validated at construction time. Extension attribute names must be 1-20 lowercase alphanumeric characters, and time must be a timezone-aware datetime.
  5. Serialize and Deserialize Kafka CloudEvents in v2

    main

    Serializing

    to_binary_event returns a frozen KafkaMessage dataclass with .headers, .key, and .value.

    from cloudevents.core.bindings.kafka import to_binary_event, KafkaMessage
    
    kafka_msg = to_binary_event(event)
    # Supports custom key mapping
    kafka_msg = to_binary_event(
        event,
        key_mapper=lambda e: e.get_extension("partitionkey"),
    )

    Deserializing

    Wrap the raw components in a KafkaMessage and use from_kafka_event to auto-detect the mode and spec version.

    from cloudevents.core.bindings.kafka import from_kafka_event, KafkaMessage
    
    msg = KafkaMessage(headers=headers, key=key, value=value)
    event = from_kafka_event(msg)
  6. Use the cloudevents.v1 compatibility layer

    main

    If you are not ready to migrate to the v2 core API, the cloudevents.v1 package provides a drop-in compatibility layer. This preserves the v1 API under a new namespace, allowing you to unpin from old top-level imports without rewriting logic.

    Note: The compat layer behaves identically to the old v1 SDK (dict-like, mutable, accepts marshaller callables) but does not enforce strict mypy or v2 validation rules.

    # Example mapping for common imports
    # Old:
    # from cloudevents.http import CloudEvent
    # New (Compat):
    from cloudevents.v1.http import CloudEvent
    
    # Old:
    # from cloudevents.conversion import to_dict
    # New (Compat):
    from cloudevents.v1.conversion import to_dict
  7. Migrate from CloudEvents SDK v1 to v2

    main

    When upgrading from v1 to v2, several core patterns and imports have changed. Key changes include the removal of Pydantic integration (use the core CloudEvent directly), the replacement of dict-like event access with explicit getter methods, and a shift in how serialization/deserialization is handled.

    Removed Features and v2 Alternatives

    Featurev1v2 Alternative
    Pydantic integrationfrom cloudevents.pydantic import CloudEventRemoved -- use the core CloudEvent directly
    Dict-like event accessevent["source"], event["x"] = yevent.get_source(), event.get_extension("x")
    from_dict()from cloudevents.http import from_dictConstruct CloudEvent(attributes=d) directly
    to_dict()from cloudevents.conversion import to_dictevent.get_attributes() + event.get_data()
    from_json()from cloudevents.http import from_jsonJSONFormat().read(None, json_bytes)
    to_json()from cloudevents.conversion import to_jsonJSONFormat().write(event)
    Custom marshallersdata_marshaller=fn / data_unmarshaller=fnImplement the Format protocol
    is_binary() / is_structured()from cloudevents.http import is_binaryMode is handled internally by from_http_event()
    Deprecated helpersto_binary_http(), to_structured_http()to_binary_event(), to_structured_event()
  8. Serialize and Deserialize HTTP CloudEvents in v2

    main

    Serializing Events

    v2 uses to_binary_event or to_structured_event which return an HTTPMessage dataclass containing .headers and .body.

    from cloudevents.core.bindings.http import to_binary_event, to_structured_event
    import requests
    
    message = to_binary_event(event)
    requests.post(url, headers=message.headers, data=message.body)

    To use a custom Format:

    from cloudevents.core.bindings.http import to_binary
    from cloudevents.core.formats.json import JSONFormat
    
    message = to_binary(event, event_format=JSONFormat())

    Deserializing Events

    First, wrap raw headers and body into an HTTPMessage, then use from_http_event to auto-detect the mode and spec version.

    from cloudevents.core.bindings.http import from_http_event, HTTPMessage
    
    message = HTTPMessage(headers=headers, body=body)
    event = from_http_event(message)

    Alternatively, you can be explicit:

    • from_binary_event(message)
    • from_structured_event(message)
  9. Understand the BaseCloudEvent protocol contract

    main
    The BaseCloudEvent is a Python Protocol that defines the standard interface for all CloudEvent implementations in this SDK. While specific implementations may provide additional functionality, any object adhering to the CloudEvent specification must implement this contract. This ensures that code written to consume CloudEvents can work with any implementation (e.g., different transport formats or specialized event types) as long as they provide these core methods.
  10. Implement a custom CloudEvent format using the Format protocol

    main

    To support a new wire format (like Avro or Protobuf), implement the Format protocol. A Format implementation is responsible for serializing and deserializing CloudEvents between their object representation and their byte representation.

    Key responsibilities include:

    • read: Deserializing a full CloudEvent from bytes/string into a BaseCloudEvent instance.
    • write: Serializing a BaseCloudEvent instance into bytes.
    • write_data: Serializing only the data payload (useful for protocol bindings like HTTP binary mode).
    • read_data: Deserializing only the data payload from a body.
    • get_content_type: Returning the appropriate Content-Type header for structured mode.