DOCSEARCHbyAlgolia
Submit Docs

wincode

repository·master·Indexed 42 days ago

https://github.com/anza-xyz/wincode

A high-performance, bincode-compatible serializer and deserializer for Rust, designed for in-place initialization and direct memory writes. It provides SchemaWrite and SchemaRead derive macros for fast serialization, an UninitBuilder to reduce boilerplate for manual SchemaRead implementations, and a Cursor for buffered I/O operations.

Tokens
17.9K
Snippets
50
Records
75
Agent score
53%

What's inside wincode

  1. Quickstart with wincode derive macros

    master

    To use wincode with your own custom struct types, use the SchemaWrite and SchemaRead derive macros. This allows you to perform fast, in-place serialization and deserialization that is compatible with bincode.

    Ensure you have the derive feature enabled in your Cargo.toml to use these macros.

    #[derive(SchemaWrite, SchemaRead)]
struct MyStruct {
    data: Vec<u64>,
    win: bool,
}

let val = MyStruct { data: vec![1,2,3], win: true };
// wincode serialization is compatible with bincode
assert_eq!(wincode::serialize(&val).unwrap(), bincode::serialize(&val).unwrap());

  2. Quickstart with wincode using derive macros

    master

    To use wincode with your own custom struct types, derive the SchemaWrite and SchemaRead traits. This allows you to perform fast, in-place serialization and deserialization that is compatible with bincode.

    wincode provides trait implementations for many built-in types such as Vec and primitive integers automatically.

    #[derive(SchemaWrite, SchemaRead)]
struct MyStruct {
    data: Vec<u64>,
    win: bool,
}

let val = MyStruct { data: vec![1,2,3], win: true };
// wincode serialization is bincode-compatible
assert_eq!(wincode::serialize(&val).unwrap(), bincode::serialize(&val).unwrap());

  5. Use top-level context with SchemaRead

    master

    You can change the trait generated by SchemaRead to SchemaReadContext by using the #[wincode(context = "<Type>")] attribute on a struct. This is useful for providing external data (like an allocator or a bump pointer) during deserialization.

    Fields marked with #[wincode(context)] will use the provided context. This allows fields to have lifetimes tied to the context rather than the input bytes, enabling data to outlive the input buffer.

    Use wincode::deserialize_with_context to perform the deserialization.

    #[cfg(all(feature = "derive", feature = "bumpalo"))] {
use bumpalo::{Bump, collections::String};
use wincode::{SchemaRead, SchemaWrite, deserialize_with_context, serialize};

#[derive(SchemaRead, SchemaWrite, Debug, PartialEq)]
#[wincode(context = "&'bump Bump")]
struct Message<'bump> {
    id: u32,
    #[wincode(context)]
    text: String<'bump>,
}

let bump = Bump::new();
let message = Message {
    id: 42,
    text: String::from_str_in("hello", &bump),
};

let bytes = serialize(&message).unwrap();
let decoded: Message = deserialize_with_context(&bump, &bytes).unwrap();
}

  6. Zero-copy characteristics of `bool`

    master

    In wincode, the bool type is optimized for zero-copy writing.

    • Writing: The in-memory byte of a bool matches its serialized form, allowing for zero-copy writes.
    • Reading: Despite zero-copy writing, reading is not zero-copy because wincode must validate that the byte is a valid boolean bit pattern (0 or 1). Any other byte value will result in a deserialization error.

  7. Use VarInt for variable-length integer encoding

    master

    The VarInt struct implements IntEncoding using a variable-length encoding scheme. This is useful for reducing serialized data size at the cost of significant performance overhead.

    Unsigned Integer Encoding Logic:

    1. If u < 251: Encoded as a single byte.
    2. If 251 <= u < 2^16: Encoded as 251 followed by a u16.
    3. If 2^16 <= u < 2^32: Encoded as 252 followed by a u32.
    4. If 2^32 <= u < 2^64: Encoded as 253 followed by a u64.
    5. If 2^64 <= u < 2^128: Encoded as 254 followed by a u128.

    Signed Integer Encoding: Signed integers are first converted to unsigned using the zigzag algorithm before being encoded as a VarInt. This ensures that values close to zero (both positive and negative) occupy fewer bytes.

    Characteristics:

    • Performance: Significantly slower than FixInt.
    • Zero-Copy: VarInt::ZERO_COPY is always false.
    • Static Size: VarInt::STATIC is always false.

  8. How TagEncoding normalizes enum discriminants

    master

    When implementing SchemaRead or SchemaWrite for enums, you cannot directly match on a generic type's discriminant using integer literals (e.g., match tag { 0 => ... }) because the compiler cannot guarantee the trait represents a closed set of integer types.

    To solve this, wincode uses the TagEncoding trait to normalize discriminants to a common u32 type. This allows you to convert the encoded tag into a u32, perform a standard integer match, and then proceed with deserialization. In release builds, this conversion is typically elided by the compiler via monomorphization, resulting in no performance penalty.

    // Example of using TagEncoding to enable integer matching in SchemaRead
unsafe impl<'de, C: Config> SchemaRead<'de, C> for Foo {
    type Dst = Self;

    fn read(reader: impl Reader<'de>, dst: &mut MaybeUninit<Self::Dst>) -> ReadResult<()> {
        // 1. Get the tag using the config's TagEncoding
        let tag_raw = C::TagEncoding::get(reader)?;
        
        // 2. Normalize to u32 to allow matching on integer literals
        let tag = C::TagEncoding::try_into_u32(tag_raw)?;

        match tag {
            0 => { /* handle Bar */ }
            1 => { /* handle Baz */ }
            _ => return Err(invalid_tag_encoding(tag as usize)),
        }

        Ok(())
    }
}

  9. Use `as_trusted_for` for high-performance I/O

    master

    The as_trusted_for method allows you to obtain a specialized Reader or Writer that bypasses certain checks, provided you can guarantee safety. This is useful for performance-critical code where you know the exact bounds of the operation.

    Safety Requirements

    • For Reader: The caller must guarantee they will not read beyond the bounds of the slice returned by the trusted reader.
    • For Writer: The caller must guarantee they will fully initialize exactly n_bytes of memory and will not write beyond the bounds of the slice.

  10. Understand `TypeMeta` for Static vs Dynamic types

    master

    Wincode uses TypeMeta to describe the nature of a type's serialized representation. This is useful for determining if a type can be handled via zero-copy or if it requires dynamic allocation.

    • TypeMeta::Static { size, zero_copy }: Used for types with a fixed, known size at compile time (e.g., most enums with uniform variants or simple structs).
      • size: The total number of bytes the type occupies when serialized.
      • zero_copy: A boolean indicating if the type can be safely read directly from a buffer without allocation. Note that even static types might have zero_copy: false if they contain discriminants that could have invalid bit patterns.
    • TypeMeta::Dynamic: Used for types whose serialized size depends on the data (e.g., enums with non-uniform variants, Vec, String, or types using varint encoding).

  11. Configure integer encoding with ByteOrder and IntEncoding

    master

    Wincode uses the ByteOrder trait and the IntEncoding trait to control how integers are serialized.

    • ByteOrder is a marker trait used to specify endianness (BigEndian or LittleEndian).
    • IntEncoding<B: ByteOrder> is an unsafe trait that provides methods for encoding, decoding, and determining the size of various integer types (u16 through u128 and i16 through i128).

    When implementing custom schema reading or writing, you can use these traits to ensure your serialization logic respects the configured byte order and supports features like ZERO_COPY if the encoding allows it.

  12. Understand BorrowKind and borrowing capabilities

    master

    The BorrowKind enum defines how data can be borrowed from a Reader. This is used to determine if a slice returned by a reader is tied to the call site or if it can extend into the reader's backing storage.

    • CallSite: Borrowed from the call site, not extending past it.
    • Backing: Borrowed from the backing store, extending past the call site.
    • BackingMut: Mutably borrowed from the backing store, extending past the call site.

    You can check if a Reader supports a specific kind of borrow using supports_borrow(kind: BorrowKind).

    // Example of checking borrow support
let reader = [1, 2, 3, 4, 5];
assert!(reader.as_slice().supports_borrow(BorrowKind::Backing));

let mut reader = [1, 2, 3, 4, 5];
assert!(reader.as_mut_slice().supports_borrow(BorrowKind::BackingMut));