eserde

repository·main·Indexed 42 days ago

https://github.com/mainmatter/eserde

A Rust library that extends serde to report multiple deserialization errors at once instead of aborting after the first failure. It uses a two-pass approach via the EDeserialize trait to accumulate schema violations like missing fields or type mismatches. Includes support for JSON and TOML via Cargo features, and provides an eserde_axum crate as a drop-in replacement for the axum::Json extractor to return RFC 9457 compliant error responses.

Tokens
7.7K
Snippets
28
Records
35
Agent score
52%

What's inside eserde

  1. How eserde improves deserialization error reporting

    main

    Standard serde deserializers (like serde_json) abort as soon as the first error is encountered. This forces API consumers into a slow feedback loop of fixing one error at a time.

    eserde solves this by performing a second pass over the input when an error occurs. It uses the eserde::EDeserialize trait to accumulate multiple errors across the entire payload, allowing you to report all schema violations (e.g., missing fields, type mismatches, or null values) to the user in a single response.

    #[derive(Debug, eserde::Deserialize)]
    struct Version {
        major: u32,
        minor: u32,
        patch: u32,
    }
    
    #[derive(Debug, eserde::Deserialize)]
    struct Package {
        version: Version,
        source: String,
    }
    
    let payload = r#"{
        "version": {
            "major": 1,
            "minor": "2"
        },
        "source": null
    }"#;
    
    // eserde::json::from_str returns all errors found in the payload
    let errors = eserde::json::from_str::<Package>(&payload).unwrap_err();
    
    // Output will include multiple errors:
    // - version.minor: invalid type: string "2", expected u32 ...
    // - version: missing field `patch` ...
    // - source: invalid type: null, expected a string ...
  2. Understand eserde performance and limitations

    main

    Because eserde aims to report multiple errors, it has different performance characteristics than standard serde:

    • Double Pass: The input must be visited twice (once for the happy path via serde, and once for error accumulation via EDeserialize).
    • No Readers: Because the input must be replayable for the second pass, you cannot deserialize directly from a non-replayable std::io::Read stream.
    • Speed: It is slower than a single-pass serde deserialization.
    • Compilation Time: #[derive(eserde::Deserialize)] generates approximately twice as much code as serde::Deserialize, which may increase compilation times.
  3. Use eserde for JSON deserialization

    main

    To use eserde with JSON, you must activate the json feature in your Cargo.toml.

    Setup:

    [dependencies]
    eserde = { version = "0.1", features = ["json"] }
    serde = "1"

    Usage:

    1. Replace #[derive(serde::Deserialize)] with #[derive(eserde::Deserialize)] on your structs.
    2. Replace serde_json::from_str with eserde::json::from_str.
    3. Replace serde_json::from_slice with eserde::json::from_slice.

    Note: eserde::json does not currently support deserializing from a reader (no from_reader equivalent).

    [dependencies]
    eserde = { version = "0.1", features = ["json"] }
    serde = "1"
    // Use eserde's JSON functions instead of serde_json
    let data: MyStruct = eserde::json::from_str(json_string)?;
  4. Install eserde

    main

    To use eserde in your Rust project, add it and serde to your Cargo.toml. Note that eserde provides format-specific support (like JSON or TOML) via Cargo features.

    [dependencies]
    eserde = { version = "0.1" }
    serde = "1"
  5. Use eserde for TOML deserialization

    main

    To use eserde with TOML, activate the toml feature in your Cargo.toml.

    Setup:

    [dependencies]
    eserde = { version = "0.1", features = ["toml"] }
    serde = "1"

    Usage: Replace toml::from_str with eserde::toml::from_str.

    [dependencies]
    eserde = { version = "0.1", features = ["toml"] }
    serde = "1"
    let config: MyConfig = eserde::toml::from_str(toml_string)?;
  6. How `Deserializer` tracks paths for error reporting

    main

    The Deserializer<D> struct is an adapter designed to wrap an existing Serde Deserializer. Its primary purpose is to record the structural path (e.g., map keys, sequence indices, enum variants) during the deserialization process. If a deserialization error occurs, this path information can be used to provide precise context about where the error happened in the data structure.

    It works by wrapping visitors and seeds with specialized types:

    • Wrap<V>: Wraps a Visitor to track structural changes like entering sequences or maps.
    • CaptureKey<X>: A specialized Deserializer and Visitor used during map key deserialization to capture the string representation of the key so it can be pushed onto the PathTracker.
    • TrackedSeed<X>: Wraps a DeserializeSeed to ensure that when seeds are used (for example, for elements in a sequence or values in a map), they use the path-aware Deserializer.
    • SeqAccess and MapAccess: Custom implementations of Serde's access traits that manage the PathTracker state by pushing segments (like Segment::Seq { index } or Segment::Map { key }) and popping them when the scope is exited.
  7. Understand JSON extraction errors in eserde_axum

    main

    When using the Json extractor in eserde_axum, several types of errors can occur during the request lifecycle. These errors are encapsulated in the JsonRejection enum. All error responses generated by these rejections follow the RFC 9457 Problem Details specification.

    Common failure scenarios include:

    1. Deserialization Failures (JsonError): Occurs when the request body is valid JSON but cannot be mapped to the target schema. The response includes specific validation errors and JSON pointers to the problematic fields.
    2. Missing Content-Type (MissingJsonContentType): Occurs when the Content-Type header is absent from the request.
    3. Incorrect Content-Type (JsonContentTypeMismatch): Occurs when the Content-Type header is present but does not match application/json or an application/*+json pattern.
    4. Body Buffering Issues (BytesRejection): Occurs when the request body cannot be read or exceeds size limits (e.g., content_too_large).
  8. Use eserde::Deserialize to collect multiple errors

    main

    Unlike serde::Deserialize, which aborts on the first error, eserde::Deserialize is designed to accumulate multiple errors in a single pass. This is useful for providing comprehensive feedback to users of an API.

    To implement this, you must:

    1. Replace #[derive(serde::Deserialize)] with #[derive(eserde::Deserialize)] on your structs.
    2. Use an eserde-specific deserialization function (like eserde::json::from_str) instead of the standard serde_json functions.
    #[derive(Debug, eserde::Deserialize)]
    struct Package {
        version: Version,
        source: String,
    }
    
    #[derive(Debug, eserde::Deserialize)]
    struct Version {
        major: u32,
        minor: u32,
        patch: u32,
    }
    
    let payload = r#"{
        "version": {
            "major": 1,
            "minor": "2"
        },
        "source": null
    }"#;
    
    // Use eserde::json instead of serde_json
    let errors = eserde::json::from_str::<Package>(&payload).unwrap_err();
    
    // errors.to_string() will contain multiple errors:
    // - version.minor: invalid type: string "2", expected u32...
    // - version: missing field `patch`...
    // - source: invalid type: null, expected a string...
  9. How JSON error accumulation works in eserde

    main

    The eserde::json module implements a two-pass deserialization strategy to provide comprehensive error reporting:

    1. First Pass: It attempts to deserialize the target type using standard serde::Deserialize. If successful, it returns Ok(value).
    2. Second Pass: If the first pass fails, eserde uses the EDeserialize trait to visit the input again. This pass is designed to traverse the entire input and accumulate as many individual DeserializationError instances as possible.
    3. Result: The errors are collected into a DeserializationErrors vector and returned in the Err variant.

    This approach allows developers to see all structural or type mismatches in a single JSON payload rather than fixing them one by one.

  10. How to use ErrorReporter when implementing a new data format

    main

    If you are implementing eserde support for a new data format, you must use the ErrorReporter to collect and report errors during deserialization. Because serde's Deserialize trait does not allow passing an error accumulator through the call stack, eserde uses a thread-local storage mechanism managed by ErrorReporter.

    To use it, you must follow this lifecycle:

    1. Initialize: Call ErrorReporter::start_deserialization() to create an ErrorReporterGuard. This initializes the PathTracker and the error buffer.
    2. Maintain Guard: You must keep the ErrorReporterGuard alive for the entire duration of the deserialization operation. If the guard is dropped, the error buffer is cleared and the operation is considered finished.
    3. Report Errors: During deserialization, call ErrorReporter::report(e) whenever an error occurs. This will capture the error along with its location in the data structure.
    4. Collect Errors: Once deserialization is complete (but while the guard is still alive), call ErrorReporter::take_errors() to retrieve the accumulated Vec<DeserializationError>.
    // Example pattern for format implementers
    let _guard = ErrorReporter::start_deserialization();
    
    // ... perform deserialization using EDeserialize::deserialize_for_errors ...
    
    let errors = ErrorReporter::take_errors();
    if !errors.is_empty() {
        // handle errors
    }
    // _guard is dropped here, cleaning up state
  11. Use the Deserialize derive macro via the eserde crate

    main

    To implement the EDeserialize trait for your types, you should not use the eserde_derive crate directly. Instead, use the Deserialize derive macro provided by the main eserde crate. This ensures that the correct version of the procedural macro is used and that all necessary internal shadow types and companion types are correctly wired.

    When you apply #[derive(Deserialize)] to a struct or enum, the macro generates:

    1. A Shadow Type: An internal representation used to perform the actual deserialization.
    2. A Companion Type: Used for error reporting and validation.
    3. An implementation of ::eserde::EDeserialize<'de> for error-aware deserialization.
    4. An implementation of ::eserde::_serde::Deserialize<'de> that uses the shadow type and then initializes the target type from it.
    use eserde::Deserialize;
    
    #[derive(Deserialize)]
    struct MyData {
        id: u64,
        name: String,
    }