ciborium

repository·main·Indexed 18 days ago

https://github.com/enarx/ciborium

A Rust implementation of CBOR (Concise Binary Object Representation) designed for the serde serialization framework. It includes ciborium-io for no_std and no_alloc I/O traits and ciborium-ll for high-performance, low-level encoding and decoding of CBOR items. The library prioritizes the smallest possible lossless numeric encoding and preserves map order using Vec<(Value, Value)>.

Tokens
12.9K
Snippets
50
Records
65
Agent score
61%

What's inside ciborium

  1. Use ciborium-io for no_std and no_alloc I/O

    main

    The ciborium-io crate provides low-level Read and Write traits designed for environments where std::io is unavailable, such as no_std or no_alloc contexts. These traits are simplified versions of the standard library counterparts and are designed to be zero-cost abstractions.

    Supported Implementations

    • Byte Slices: Always supported.
    • std::io types: Supported if the std feature is enabled (via blanket implementations).
    • Vec<u8>: Supported if the alloc feature is enabled.
    • Custom Types: You can implement the Read and Write traits for your own types to integrate with the rest of the ciborium ecosystem.
  2. Use ciborium-ll for low-level CBOR parsing

    main
    ciborium-ll provides low-level types for encoding and decoding CBOR items. It is designed for high-performance, low-level manipulation and is compatible with both no_std and no_alloc environments. The crate operates by providing Decoder and Encoder types that handle CBOR Header instances, allowing you to either manage the item bodies manually or use provided utility functions.
  3. Encode CBOR items with Encoder

    main

    To encode values, create an Encoder from a writer. Use Encoder::push() to write a Header to the wire. After writing the header, you can write the item body directly.

    Utility functions:

    • Encoder::bytes(): Properly segments byte output on the wire.
    • Encoder::text(): Properly segments text output on the wire.
    • Encoder::flush(): Finalizes the encoding process.
    use ciborium_ll::{Encoder, Header};
    use ciborium_io::Write as _;
    
    let mut buffer = [0u8; 19];
    let mut encoder = Encoder::from(&mut buffer[..]);
    
    // Write the structure
    encoder.push(Header::Map(Some(1))).unwrap();
    encoder.push(Header::Positive(7)).unwrap();
    encoder.text("Hello, World!", 7).unwrap();
    
    // Validate our output
    encoder.flush().unwrap();
    assert_eq!(b"\xa1\x07\x7f\x67Hello, \x66World!\xff", &buffer[..]);
  4. How Ciborium represents Maps

    main

    Unlike many other serde parsers that use BTreeMap or HashMap for map types, Ciborium represents the Map type using Vec<(Value, Value)>.

    This design choice ensures that the order of key-value pairs is preserved exactly as they appear on the wire. If you require the properties of a BTreeMap or HashMap, you can convert the resulting vector using .collect().

  5. Decode CBOR items with Decoder

    main

    To decode CBOR, create a Decoder from a reader. You can use Decoder::pull() to retrieve Header instances from the input.

    Handling different item types:

    • Simple items: Most items are fully contained in their headers and can be evaluated directly from the Header instance.
    • Bytes and Text: These items have bodies that may be segmented. Use the helper functions Decoder::bytes() and Decoder::text() to parse them correctly.
    • Arrays and Maps: These contain child items. You can parse them by repeatedly calling Decoder::pull() to consume the child items within the body.
    use ciborium_ll::{Decoder, Header};
    use ciborium_io::Read as _;
    
    let input = b"\x6dHello, World!";
    let mut decoder = Decoder::from(&input[..]);
    
    match decoder.pull().unwrap() {
        Header::Text(len) => {
            let mut segments = decoder.text(len);
            // Iterate through segments and pull chunks from them
        }
        _ => panic!("unexpected value"),
    }
  6. How Ciborium handles numeric serialization

    main

    Ciborium always serializes numeric values to the smallest possible lossless encoding.

    For example, 1u128 is serialized as a single byte (01). During decoding, these small encodings are losslessly coerced back into the target type (e.g., a u128).

    For floating-point numbers, coercion to a smaller size only occurs if the raw bits remain identical when coerced back to the original size. This approach ensures maximum compatibility with dynamic languages like Python that do not use fixed integer widths.

  7. Compatibility and the Robustness Principle

    main

    Ciborium follows the Robustness Principle: it aims to be liberal in what it accepts.

    • Decoding: Ciborium aims to be wire-compatible with other implementations. For example, it can successfully decode encodings produced by serde_cbor, even though serde_cbor uses fixed-width encoding and does not perform lossless coercion.
    • Encoding: Ciborium may not necessarily be wire-compatible with other implementations when encoding, due to its preference for smallest-size numeric serialization.
  8. Quick Start with Ciborium serialization and deserialization

    main

    Ciborium provides CBOR (Concise Binary Object Representation) implementations for serde.

    To deserialize CBOR data from a reader, use from_reader(). To serialize data into a writer, use into_writer().

    Both functions accept any type that implements Read or Write, including byte slices (&[u8]) or streams. For working with dynamic CBOR values without a predefined schema, use the Value type.

    // Note: The following is a conceptual usage pattern based on the documentation
    // as exact imports depend on the crate structure.
    
    // Deserialization
    let data: MyStruct = ciborium::de::from_reader(reader).unwrap();
    
    // Serialization
    ciborium::ser::into_writer(my_instance, writer).unwrap();
    
    // Dynamic values
    let value: ciborium::value::Value = ciborium::de::from_reader(reader).unwrap();
  9. Process segmented bytes and text

    main

    CBOR supports segmented bytes and text. To handle these correctly, ciborium-ll provides bytes() and text() methods that return a Segments iterator.

    To use them correctly:

    1. Call decoder.pull() to get the header.
    2. If the header is Header::Bytes(len) or Header::Text(len), call decoder.bytes(len) or decoder.text(len) respectively.
    3. You must pass the len obtained from the header into these methods.

    These methods encapsulate the logic for navigating segmented data, which can be complex to implement manually.

    // Example for segmented bytes
    if let Header::Bytes(len) = decoder.pull()? {
        let segments = decoder.bytes(Some(len));
        for segment in segments {
            // process segment
        }
    }
  10. Represent dynamic CBOR data with the `Value` enum

    main

    The Value enum is a dynamic representation of CBOR data. It can hold various types including integers, bytes, floats, text, booleans, null, tags, arrays, and maps. You can create Value instances using the From trait for most standard Rust types.

    use ciborium::Value;
    
    let val = Value::Text("hello".to_string());
    let arr = Value::Array(vec![Value::Bool(true), Value::Null]);
    let map = Value::Map(vec![(Value::Text("key".into()), Value::Integer(17.into()))]);
  11. Access and transform `Value` data

    main

    The Value enum provides several methods to inspect and extract its underlying data. Methods generally follow these patterns:

    • is_<type>(): Returns true if the value matches the type.
    • as_<type>(): Returns an Option<&T> (immutable reference) or Option<&mut T> (mutable reference) if the type matches.
    • into_<type>(): Consumes the Value and returns Result<T, Value>, returning the underlying data if successful or the original Value in an Err if the type mismatch occurs.