fxamacker/cbor

repository·master·Indexed 21 days ago

https://github.com/fxamacker/cbor

A high-performance, secure Go implementation of the CBOR (Concise Binary Object Representation) standard (IETF STD 94 / RFC 8949) and CBOR Sequences (RFC 8742). It provides an API similar to encoding/json, supports Extended Diagnostic Notation, and includes configurable limits to protect against resource exhaustion attacks. Key features include full conformance to IETF standards, data size optimization via struct tags, and support for custom encoding/decoding modes and CBOR tags.

Tokens
11.4K
Snippets
41
Records
64
Agent score
77%

What's inside fxamacker/cbor

  1. Overview of fxamacker/cbor

    master

    fxamacker/cbor

    fxamacker/cbor is a high-performance Go library for encoding and decoding CBOR (IETF STD 94 / RFC 8949) and CBOR Sequences (RFC 8742). It also supports Extended Diagnostic Notation (Appendix G of RFC 8610).

    Key Features:

    • Full Conformance: Complies with IETF STD 94.
    • Security: Includes configurable limits to defend against malicious/adversarial CBOR data, preventing resource exhaustion attacks (unlike encoding/gob).
    • Performance: Fast encoding/decoding without using Go's unsafe package. Rejects malformed data very efficiently.
    • Data Size Optimization: Supports struct tags to reduce encoded size and can optionally shrink float64 to float32 or float16 if values fit.
    • Usability: The API is designed to be similar to encoding/json, making it easy to adopt. Encoding and decoding modes can be created at startup and reused across goroutines.
  2. CBOR Standards and Feature Support

    master

    fxamacker/cbor is a CBOR codec that provides full conformance with IETF STD 94 (RFC 8949). It also supports CBOR Sequences (RFC 8742) and Extended Diagnostic Notation (Appendix G of RFC 8610).

    Key Features:

    • CBOR tags: Supports both built-in and user-defined tags.
    • Preferred serialization: Integers are encoded using the fewest possible bytes. Supports optional float64 → float32 → float16 downcasting.
    • Map key sorting: Supports Unsorted, length-first (Canonical CBOR), and bytewise-lexicographic (CTAP2) modes.
    • Duplicate map keys: Encoding always forbids duplicate keys. Decoding can be configured to allow or forbid them.
    • Indefinite length data: Configurable options for both encoding and decoding.
    • Security: Implements protections against integer overflow and resource exhaustion as per RFC 8949 Section 10.
    • Well-formedness: All data is checked for well-formedness and syntax errors.
  3. Handling Duplicate Map Keys during Decoding

    master

    The decoder provides different strategies for managing duplicate map keys in a CBOR map, depending on whether you prioritize speed or strictness:

    • DupMapKeyQuiet: Disables detection of duplicate map keys to maximize performance. It uses a "keep fastest" approach, choosing either "keep first" or "keep last" based on the target Go data type.
    • DupMapKeyEnforcedAPF: Enforces strict rejection of duplicate keys. If a duplicate is detected, decoding stops immediately and returns a DupMapKeyError.
      • APF (Allow Partial Fill): The suffix indicates that the destination map or struct may contain some partially decoded values at the time the error is returned. It is the caller's responsibility to discard these results if the protocol requires strict atomicity.
  4. Implement custom Marshaler/Unmarshaler for CBOR tags

    master

    You can support any CBOR tag number by implementing the cbor.Marshaler and cbor.Unmarshaler interfaces. When these methods (MarshalCBOR and UnmarshalCBOR) are implemented, the codec will automatically call them during standard Marshal or Unmarshal operations.

    // Example: Implementing custom logic for a specific tag
    func (v EmbeddedJSON) MarshalCBOR() ([]byte, error) {
        // ... custom logic to wrap data in a cbor.Tag ...
    }
    
    func (v *EmbeddedJSON) UnmarshalCBOR(b []byte) error {
        // ... custom logic to extract data from a cbor.Tag ...
    }
  5. Create and reuse Custom Modes with Presets

    master

    For specific requirements (like WebAuthn/CTAP2), you should create custom encoding and decoding modes. Modes are created from settings (EncOptions or DecOptions), are immutable once created, and are safe for concurrent use. It is recommended to create modes at startup and reuse them.

    Available Presets:

    • CoreDetEncOptions(): RFC 8949 Core Deterministic Encoding
    • PreferredUnsortedEncOptions(): RFC 8949 Preferred Serialization
    • CanonicalEncOptions(): RFC 7049 Canonical CBOR
    • CTAP2EncOptions(): FIDO2 CTAP2 Canonical CBOR
    // Create encoding mode.
    opts := cbor.CoreDetEncOptions()   // use preset options as a starting point
    opts.Time = cbor.TimeUnix          // change any settings if needed
    em, err := opts.EncMode()          // create an immutable encoding mode
    
    // Reuse the encoding mode. It is safe for concurrent use.
    
    // API matches encoding/json.
    b, err := em.Marshal(v)            // encode v to []byte b
    encoder := em.NewEncoder(w)         // create encoder with io.Writer w
    err := encoder.Encode(v)            // encode v to io.Writer w
  6. Tag Validity and Decoding Behavior

    master

    The library performs validity checks on built-in tags (currently 0, 1, 2, 3, and 55799) to ensure the tag content matches the expected type and value.

    Unknown Tags

    When encountering unknown tag data items:

    • Decoding into any: The item is decoded into a cbor.Tag type, which contains the tag number and the decoded tag content.
    • Decoding into specific Go types: The item is decoded into the specified Go type. If that Go type is registered with a specific tag number, the tag number can optionally be verified.

    Forbidding Tags

    Some protocols (like CTAP2 Canonical CBOR) require forbidding all tag data items. The decoder provides an option to treat any tag data item as an error.

  7. Reduce encoded size using Struct Tag options

    master

    You can use Go struct tags to automatically reduce the size of encoded data and improve performance.

    Supported Struct Tags:

    • toarray: Encodes the struct without field names (as an array). Note: When using toarray, the encoder ignores omitempty and omitzero to maintain fixed element positions for correct decoding.
    • keyasint: Encodes field names as integers instead of strings.
    • omitempty: Omits the field when it contains its zero value.
    • omitzero: Omits the field when it contains its zero value.
    • -: Omits the field entirely from encoding.

    Using these tags can significantly reduce the footprint of your data (e.g., reducing a nested struct to just 1 byte of CBOR compared to 18 bytes of JSON).

    type GrandChild struct {
    	Quux int `cbor:"omitempty"` // Note: use cbor tag prefix for this library
    }
    
    type Child struct {
    	Baz int        `cbor:"omitempty"` 
    	Qux GrandChild `cbor:"omitempty"` 
    }
    
    type Parent struct {
    	Foo Child `cbor:"omitempty"` 
    	Bar int   `cbor:"omitempty"` 
    }
  8. Install fxamacker/cbor/v2

    master

    To use this library in your Go project, install it using go get and import the v2 module.

    go get github.com/fxamacker/cbor/v2

    Note for TinyGo users: If you are using TinyGo, you may need to use the experimental branch feature/cbor-tinygo-beta for compatibility. This branch requires TinyGo v0.42.0-dev-801bd484 or newer.

  9. Quick Start: Encoding and Decoding

    master

    The fxamacker/cbor API is largely compatible with the standard encoding/json patterns. You can use cbor.Marshal to encode data and cbor.Unmarshal to decode it. For human-readable inspection of encoded bytes, you can use cbor.Diagnose to get the Extended Diagnostic Notation (DN).

    package main
    
    import (
    	"encoding/hex"
    	"fmt"
    
    	"github.com/fxamacker/cbor/v2"
    )
    
    type GrandChild struct {
    	Quux int `cbor:"omitempty"` 
    }
    
    type Child struct {
    	Baz int        `cbor:"omitempty"` 
    	Qux GrandChild `cbor:"omitempty"` 
    }
    
    type Parent struct {
    	Foo Child `cbor:"omitempty"` 
    	Bar int   `cbor:"omitempty"` 
    }
    
    func main() {
    	// Encoding
    	results, _ := cbor.Marshal(Parent{})
    	fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
    
    	// Diagnostic Notation (Human readable)
    	text, _ := cbor.Diagnose(results)
    	fmt.Println("DN: " + text)
    }
  10. Use indefinite-length encoding for arrays, maps, and strings

    master

    The Encoder supports indefinite-length encoding, which allows you to write a sequence of items until a "break" code is sent. This is useful for streaming data of unknown size.

    Supported types:

    • StartIndefiniteArray(): For indefinite-length arrays.
    • StartIndefiniteMap(): For indefinite-length maps. Note that maps must contain an even number of items (key-value pairs) before closing.
    • StartIndefiniteByteString(): For indefinite-length byte strings.
    • StartIndefiniteTextString(): For indefinite-length text strings.

    Closing indefinite-length items: Call EndIndefinite() to write the CBOR "break" code and close the current indefinite-length container. If you call EndIndefinite() on a map with an odd number of items, it returns an IndefiniteLengthMapOddItemCountError and does not write the break code, allowing you to add the missing item and retry.

    enc := cbor.NewEncoder(w)
    
    // Encoding an indefinite array
    enc.StartIndefiniteArray()
    enc.Encode(1)
    enc.Encode("hello")
    enc.EndIndefinite()
  11. Configure CBOR encoding with Struct Tags

    master

    The library supports standard cbor struct tags (e.g., cbor:"name,omitempty"). If both cbor and json tags are present, the cbor tag takes precedence.

    Specialized options to reduce encoding size include:

    • keyasint: Encodes a field as an element of a CBOR map with a specified integer key.
    • toarray: Encodes struct fields as elements of a CBOR array instead of a map.
    • omitempty: Omits the field if it has a zero value.
    • omitzero: Omits the field if it has a zero value (matching encoding/json behavior).
  12. Configure Tag handling with TagOptions

    master

    You can control how the encoder and decoder treat CBOR tags using TagOptions. This is useful when you want to ignore tags, make them optional, or require them for specific types.

    DecTagMode (Decoder settings)

    • DecTagIgnored: The decoder skips the tag number if present.
    • DecTagOptional: The decoder verifies the tag number if it is present.
    • DecTagRequired: The decoder verifies the tag number and requires it to be present.

    EncTagMode (Encoder settings)

    • EncTagNone: The encoder does not encode a tag number.
    • EncTagRequired: The encoder encodes the tag number.
    opts := cbor.TagOptions{
        DecTag: cbor.DecTagRequired,
        EncTag: cbor.EncTagRequired,
    }