DOCSEARCHbyAlgolia
Submit Docs

Chronicle Wire Documentation

repository·ea·Indexed 20 days ago

https://github.com/openhft/chronicle-wire

A high-performance, zero-GC serialization library providing a common API for multiple wire formats, including YAML, JSON, CSV, and various binary formats. It supports low-latency applications through features like the 'events by method' pattern for asynchronous communication, BytesMarshallable for direct buffer access, and a range of encoding strategies from human-readable text to compact, field-less binary formats.

Tokens
32K
Snippets
60
Records
135
Agent score
69%

What's inside Chronicle Wire

  1. Overview of Chronicle Wire features and use cases

    ea

    Chronicle Wire is a high-performance, zero-GC serialization library that abstracts underlying wire formats. It allows you to use human-readable text (YAML, JSON, CSV) or compact binary formats interchangeably using a common API.

    Key Features:

    • Multiple wire formats (YAML, JSON, CSV, binary variants).
    • Schema evolution (handling optional fields, different field orders, or unexpected fields).
    • Low-latency, low-allocation serialization of Marshallable POJOs.
    • Automatic conversion between formats.
    • Support for hybrid wire formats (embedding one format within another).

    Typical Use Cases:

    • Configuration files with aliased type information.
    • Persistence of in-memory state.
    • Extremely fast IPC (Inter-Process Communication) between JVMs.

  2. Compare Chronicle Wire formats with other serialization libraries

    ea

    The microbenchmarks provide a comparison of various serialization formats based on their encoded size (in characters or bytes).

    Key comparisons include:

    • Binary Formats: RawWire, BytesMarshallable (with and without stop bit encoding), BSON, and SBE (Simple Binary Encoding).
    • Text/JSON Formats: SnakeYAML, Boon, and Jackson.
    • Java Native: Externalizable (noted as being more efficient than Serializable but still heavyweight compared to Chronicle Wire formats).

  4. What is Chronicle Wire and its core concepts?

    ea

    Chronicle Wire is a high-performance, low-latency Java library for serialization and deserialization. It is designed to be format-agnostic, meaning you can write your serialization logic once (e.g., using Marshallable objects) and switch between formats like YAML, JSON, or Binary by simply changing the WireType without altering application code.

    Key Abstractions

    • Bytes: A core abstraction (from chronicle-bytes) representing a sequence of bytes. It can wrap on-heap byte arrays, off-heap direct memory, or memory-mapped files. All Wire implementations read from and write to Bytes.
    • Wire: The primary interface for reading and writing structured data in a specific format. You obtain a Wire by applying a WireType to a Bytes buffer.
    • WireType: An enum that defines supported serialization formats (e.g., YAML_ONLY, JSON_ONLY, BINARY_LIGHT, FIELDLESS_BINARY) and acts as a factory for creating Wire instances.
    • DocumentContext: Manages the boundaries of individual messages or "documents" when streaming multiple items. It is crucial for ensuring messages are correctly framed, especially in binary formats or when used with Chronicle Queue. Use it with try-with-resources to ensure proper closing (which finalizes documents for writing or releases resources for reading).

  5. Chronicle Wire integration with the OpenHFT ecosystem

    ea

    Chronicle Wire is a foundational component used across the OpenHFT ecosystem:

    • Chronicle Queue: Serializes messages using MethodWriter/Reader into Queue Excerpts.
    • Chronicle Map: Serializes keys and values for storage in off-heap maps.
    • Chronicle Network: Marshals data for network communication.
    • Chronicle Services: Defines and serializes events for event-driven architectures.

  6. Compose events using chained method calls

    ea

    You can use interface inheritance to create fluent APIs for composing events. This allows you to prepend metadata like routing information or timestamps to an underlying event without changing the core event's API.

    Example Pattern:

    1. Define a base event (e.g., Saying).
    2. Define wrapper interfaces (e.g., Timed<T>, Destination<T>) that return the generic type T.
    3. Create a combined interface (e.g., DestinationTimedSaying).

    Usage:

    DestinationTimedSaying writer = wire.methodWriter(DestinationTimedSaying.class);
writer.via("targetQueue")
      .at(System.nanoTime())
      .say("Hello Composed World");
    interface Saying {
    void say(String hello);
}
interface Timed<T> {
    T at(@LongConversion(NanoTimestampLongConverter.class) long time);
}
interface TimedSaying extends Timed<Saying> { }
interface Destination<T> {
    T via(String via);
}
interface DestinationTimedSaying extends Destination<TimedSaying> { }

// Usage
DestinationTimedSaying writer = wire.methodWriter(DestinationTimedSaying.class);
writer.via("targetQueue")
      .at(System.nanoTime())
      .say("Hello Composed World");

  7. Core Interfaces in Chronicle Wire

    ea

    Chronicle Wire provides several key interfaces that act as integration points for consumers. The primary interfaces are:

    • net.openhft.chronicle.wire.Wire: The central entry point for creating document contexts, exposing WireIn/WireOut, and managing format-specific features.
    • net.openhft.chronicle.wire.DocumentContext: Manages the scope of a single message, tracks metadata (such as rolling cycle and chain ID), and coordinates the release of the document. Crucial: Always use try-with-resources to close this context.
    • net.openhft.chronicle.wire.MethodWriter / MethodReader: Used to generate strongly-typed service proxies and decoders that operate on top of Wire documents.
    • net.openhft.chronicle.wire.converter.LongConverter: An SPI for mapping textual encodings to numeric values without intermediate allocation.

  8. How to implement Marshallable objects

    ea

    To enable serialization/deserialization for your POJOs, you can implement one of the following interfaces:

    • Marshallable: The standard interface requiring readMarshallable(WireIn) and writeMarshallable(WireOut) methods.
    • SelfDescribingMarshallable: An abstract base class that implements Marshallable. It automatically provides serialization/deserialization logic for fields (via reflection or code generation) and generates toString(), equals(), and hashCode().
    • BytesMarshallable: A low-level interface for maximum performance. Implementors read/write directly from/to BytesIn/BytesOut, bypassing higher-level Wire mechanisms.

  9. Handle schema evolution by adding new fields

    ea

    Chronicle Wire's self-describing formats (YAML, JSON, BINARY_LIGHT) support backward and forward compatibility when adding fields:

    • New code reading old data: If a new field is added to a class, it will be initialized to its Java default value (e.g., 0 for int, null for objects) when reading older data that lacks that field.
    • Old code reading new data: Older versions of a class will simply ignore unrecognized new fields present in the data stream.

    To ensure smooth mapping when using type information (like !ClassName), use ClassAliasPool to assign consistent aliases to your classes.

  10. Understand Scalar and Collection Semantics

    ea

    Chronicle Wire applies specific rules to how data types are encoded:

    • Numeric Types: Encoded using the narrowest width that preserves precision. Note that long timestamps must be kept in epoch nanoseconds for compatibility with Pauser and engine components.
    • Text: Values are encoded as UTF-8 bytes on the wire.
    • Collections: Written as nested documents. Arrays use implicit numeric keys, while maps use explicit field names. Ordered collections preserve their insertion order.

  11. Serialize objects using Marshallable

    ea

    To enable automatic serialization of POJOs, implement the Marshallable interface.

    • Marshallable: Allows Chronicle Wire to automatically serialize fields. You can implement readMarshallable(WireIn wire) and writeMarshallable(WireOut wire) for manual control.
    • SelfDescribingMarshallable: A common base class that provides default implementations for marshalling, as well as toString(), equals(), and hashCode() based on the serialized form.
    • BytesMarshallable: A lower-level interface for objects that need direct, high-performance control over byte-level serialization, bypassing the standard Wire abstraction.

    For maximum performance, Chronicle Wire can use code generation to create specialized bytecode for Marshallable classes, avoiding reflection overhead.

  12. YAML 1.2.2 Specification Compliance in Chronicle Wire

    ea

    Chronicle Wire implements YAML parsing primarily through YamlWire and TextWire. The YAML wire type aims for high compliance with the YAML 1.2.2 specification, focusing on features necessary for robust data serialization and real-world usage. While TEXT wire type supports a narrower subset, YamlWire handles complex structures like anchors, aliases, and custom tags.

    Supported Features

    • Document Boundaries: Supports Document Start (---). Document End (...) is recognized but not strictly required if followed by another --- or EOF.
    • Comments: # comments are parsed and ignored. You can use the @Comment annotation to generate comments during serialization.
    • Anchors & Aliases: Supports & (anchors) and * (aliases) for repeated nodes, object graphs with cycles, or shared references.
    • Tags: Supports standard (e.g., !!map) and custom tags (e.g., !mytype). !!binary is compatible with BytesStore and byte[].
    • Scalars:
      • Literal Block Scalars (|): Supported; newlines are preserved.
      • Folded Block Scalars (>): Supported (partial compliance regarding whitespace).
      • Quoted Scalars: Single (') and double (") quotes are supported; escape sequences are handled in double quotes.
      • Plain Scalars: Most unquoted scalars are supported.
    • Numbers: Supports decimal, hexadecimal (0x), and octal (0o) integers.
    • Timestamps: Supports ISO-8601 timestamps, which can be mapped to @NanoTime or @MillisTime.

    Deviations and Limitations

    • Complex Mapping Keys: Not supported. Mapping keys must be scalars; the explicit ? key : value syntax is not supported.
    • Unordered Sets (!!set): Deserialized as LinkedHashMap or List. You must convert these to a Set in your application code if required.
    • Ordered Mappings (!!omap): Deserialized as LinkedHashMap (which preserves insertion order).
    • Floating-Point: Parsing of special values like .inf, -.inf, and .nan is limited.
    • Multi-line Flow Scalars: May require quoting to ensure correct parsing.