repository·main·Indexed 23 days ago
https://github.com/simd-lite/simd-jsonA high-performance Rust port of the simdjson JSON parser, version 0.17.3. It utilizes SIMD instructions and provides Serde compatibility for fast JSON parsing and deserialization. The library features a Tape-based DOM structure, reusable Buffers to minimize allocations, and AlignedBuf for SIMD-aligned memory management. It supports multiple architectures including SSE4.2, AVX2, NEON, and WASM SIMD128.
When writing code for this project, adhere to the following standards:
serde for JSON serialization and deserialization.Before contributing to the Rust port of simd-json, ensure your environment meets the following requirements:
example folder in the repository to understand the project's goals and usage patterns.Comprehensive test coverage is required for all contributions. You must:
To contribute code to simd-json, follow these steps:
feature/your-feature).serde for JSON serialization/deserialization.data folder for additional information during development.git clone https://github.com/your-username/your-repo.git
git checkout -b feature/your-feature
# ... write code and tests ...
git push origin feature/your-featureThe Value type provides a lazy implementation of the JSON DOM. It starts as a Tape variant, which allows for extremely cheap parsing and data access by deferring full deserialization.
As long as you only perform non-mutating operations, the value remains a Tape. If you perform a mutating operation (like insert), the Value automatically "upgrades" itself to a borrowed Value (using a Cow internally) to support the changes. This pattern allows you to benefit from high-performance lazy access while still maintaining the ability to modify the JSON structure when necessary.
use simd_json::{prelude::*, value::lazy::Value};
let mut json = br#"{"key": "value", "snot": 42}"# .to_vec();
let tape = simd_json::to_tape( json.as_mut_slice()).unwrap();
let value = tape.as_value();
let mut lazy = Value::from_tape(value);
assert_eq!(lazy.get("key").unwrap(), "value");
assert!(lazy.is_tape());
// Mutating the value triggers an upgrade from Tape to Value
lazy.insert("new", 42);
assert!(lazy.is_value());
assert_eq!(lazy.get("key").unwrap(), "value");
assert_eq!(lazy.get("new").unwrap(), 42);The library provides two primary DOM (Document Object Model) implementations for interacting with parsed JSON:
BorrowedValue: Uses &str for strings, referencing the original input slice. While it does not perform full zero-copy parsing (because it must de-escape strings in-place), it avoids allocating new memory for string content. This is ideal for performance when the input buffer can be mutated and its lifetime is managed.OwnedValue: A 'lifetimeless' version where each string value allocates a new String. Use this when you need to move the parsed data across different parts of your application without worrying about the original input buffer's lifetime.Both implementations can be interacted with using the value_trait methods for inspection and mutation.
The Value type provides an owned JSON DOM that allows for easy access and modification of data using indexing. It is slower than BorrowedValue but avoids lifetime complexities.
usize indexing to access or mutate array elements.&str indexing to access or mutate object values by key.Note: Indexing with [] will panic if the index is out of bounds or the key does not exist. For safe access, use the ValueTrait methods (like as_array, as_object, etc.) or the get methods provided by the traits.
use simd_json::{OwnedValue, json};
use simd_json::prelude::*;
// Access via array indexes
let mut a = json!([1, 2, 3]);
assert_eq!(a[1], 2);
a[1] = 42.into();
assert_eq!(a[1], 42);
// Access via object keys
let mut b = json!({"key": "not the value"});
assert_eq!(b["key"], "not the value");
b["key"] = "value".into();
assert_eq!(b["key"], "value");When using OwnedValue, you can treat JSON objects like HashMaps and JSON arrays like Vectors. This allows for easy creation, insertion, and nested mutation.
Object usage:
v.insert(key, value): Inserts a key-value pair.v.get(key): Returns an option to the value.v.remove(key): Removes a key and returns its value.v[key]: Indexing access.Array usage:
v.push(value): Appends an element.v.pop(): Removes and returns the last element.v.get_idx(index): Returns an option to the element at the specified index.v[index]: Indexing access.The Value<'value> type is a borrowed DOM. It is designed for maximum performance by referencing the original input byte slice instead of allocating new memory for strings and keys.
Cow<'value, str> to point into the input buffer.Value is tied to the lifetime of the input buffer. You cannot return a Value<'a> from a function if the buffer was created inside that function.&mut [u8] because simd-json performs in-place de-escaping of strings.Value<'value> (Borrowed) when you are parsing data immediately and want to minimize overhead.into_static() or convert to an owned representation when you need the JSON data to outlive the input buffer.Use simd-json to deserialize JSON directly into Rust structs or types that implement serde::Deserialize.
Important Performance Note: If you are parsing into a DOM (Document Object Model), you should use to_owned_value or to_borrowed_value instead of Serde, as they provide significantly better performance. Use Serde primarily when parsing directly to specific structs or when required by other parts of your application.
Note on Mutability: Most deserialization functions in simd-json require a mutable slice (&mut [u8]) or string (&mut str) because the parser rewrites the input in-place to improve performance. This may result in the input no longer being valid UTF-8 after the call.
To extract primitive values from an Owned Value, use the methods provided by the ValueAsScalar trait. These methods return an Option if the type matches or if the value is a StaticNode containing the requested type.
Supported scalar conversions:
as_null() -> Option<()>as_bool() -> Option<bool>as_i64() -> Option<i64>as_i128() -> Option<i128>as_u64() -> Option<u64>as_u128() -> Option<u128>as_f64() -> Option<f64>as_str() -> Option<&str>You can query the Deserializer to determine which SIMD implementation or architecture-specific algorithm is currently being used. This is useful for performance profiling or debugging.
Supported Implementation variants include:
Native: Standard Rust implementation.StdSimd: Using std::simd (portable SIMD).SSE42: Intel/AMD SSE4.2.AVX2: Intel/AMD AVX2.NEON: ARM NEON.SIMD128: WebAssembly SIMD128.