Overview of yyjson
masterint64, uint64, and double numbers, and offers features like JSON5 support and custom allocators.repository·master·Indexed 26 days ago
https://github.com/ibireme/yyjsonA high-performance, ANSI C-compliant (C89) JSON library designed for rapid parsing and serialization of large JSON datasets. It strictly complies with RFC 8259 and supports JSON5 features, custom allocators, and accurate reading/writing of int64, uint64, and double numbers. The library provides both immutable structures for reading and mutable structures for building JSON, along with an incremental reading API for handling very large documents without freezing applications.
int64, uint64, and double numbers, and offers features like JSON5 support and custom allocators.yyjson uses two distinct types of data structures depending on whether you are reading or building JSON:
yyjson_doc and yyjson_val): Returned when reading an existing JSON document. These cannot be modified after creation.yyjson_mut_doc and yyjson_mut_val): Used when building a new JSON document from scratch. These can be modified (e.g., appending or removing elements).While yyjson provides functions to convert between these types, it is recommended to use the public API rather than accessing these structs directly.
yyjson uses two types of data structures depending on whether you are reading or building JSON:
| Type | Immutable (Reading) | Mutable (Building) |
|---|---|---|
| Document | yyjson_doc | yyjson_mut_doc |
| Value | yyjson_val | yyjson_mut_val |
Key Concepts:
yyjson_doc_mut_copy (doc $\rightarrow$ mut_doc), yyjson_val_mut_copy (val $\rightarrow$ mut_val), yyjson_mut_doc_imut_copy (mut_doc $\rightarrow$ doc), and yyjson_mut_val_imut_copy (mut_val $\rightarrow$ val).mut after the prefix (e.g., yyjson_is_str $\rightarrow$ yyjson_mut_is_str).To improve reading speed, use the YYJSON_READ_INSITU flag. This allows the reader to use the input buffer itself to store string values.
Critical Requirements:
YYJSON_PADDING_SIZE bytes (typically 4).yyjson_doc_free() has been called.size_t dat_len = ...;
char *buf = malloc(dat_len + YYJSON_PADDING_SIZE); // create a buffer larger than (len + 4)
read_from_socket(buf, ...);
memset(buf + dat_len, 0, YYJSON_PADDING_SIZE); // set 4-byte padding after data
yyjson_doc *doc = yyjson_read_opts(buf, dat_len, YYJSON_READ_INSITU, NULL, NULL);
if (doc) {...}
yyjson_doc_free(doc);
free(buf); // the input data should be freed after the document.To build yyjson without a libc (e.g., for WebAssembly targets without a libc sysroot), define YYJSON_FREESTANDING as 1.
When using this mode:
string.h functions like memcpy, memmove, memset, memcmp, and strlen are provided via built-in inline fallbacks.FILE pointer APIs are disabled (equivalent to YYJSON_DISABLE_FILE).malloc and free are unavailable. You must either pass a yyjson_alc allocator to compatible APIs or define a global default at compile time using -DYYJSON_CUSTOM_ALC=my_alc.YYJSON_FREESTANDING_HEADER to point to your custom header.The standard public APIs perform null checks on every input parameter to prevent crashes. If you are certain a value is non-null and matches the expected type (e.g., when iterating over a known valid JSON structure), you can use the unsafe_ prefix API to improve performance.
Example:
size_t idx, max;
yyjson_val *key, *val;
yyjson_obj_foreach(obj, idx, max, key, val) {
// using unsafe_ prefix to skip checks
if (unsafe_yyjson_equals_str(key, "id") &&
unsafe_yyjson_is_uint(val) &&
unsafe_yyjson_get_uint(val) == 1234) {
...
}
}When a reading function fails, pass a pointer to a yyjson_read_err struct to receive details.
Error Details:
err.msg: Error message string.err.code: Error code (yyjson_read_code).err.pos: Byte position where the error occurred.Locating Line and Column:
Use yyjson_locate_pos() to convert the byte position into human-readable line and column numbers. Note that line and column start from 1, while character starts from 0.
char *dat = ...;
size_t dat_len = ...;
yyjson_read_err err = ...;
// 1. Check error details
if (!doc) {
printf("read error: %s, code: %u at byte position: %lu\n",
err.msg, err.code, err.pos);
}
// 2. Get line/column/char
size_t line, col, chr;
if (yyjson_locate_pos(dat, dat_len, err.pos, &line, &col, &chr)) {
printf("error at line: %lu, column: %lu, character index: %lu\n",
line, col, chr);
}char *dat = ...;
size_t dat_len = ...;
yyjson_read_err err = ...;
if (!doc) {
printf("read error: %s, code: %u at byte position: %lu\n",
err.msg, err.code, err.pos);
// printed:
// read error: trailing comma is not allowed, code: 7, at byte position: 40
}
yyjson_doc_free(doc);Memory management in yyjson is document-centric. A JSON document (yyjson_doc or yyjson_mut_doc) owns the memory for all its contained values and strings.
yyjson_doc_free() for immutable documents or yyjson_mut_doc_free() for mutable documents when they are no longer needed.yyjson_val or yyjson_mut_val) share the lifetime of their parent document. You cannot free a value independently; it will be freed automatically when the document is freed.To prevent freezing the program when reading very large documents, use the incremental reading API. This is slightly slower than standard reading but keeps the application responsive.
Note: The incremental reader only supports standard JSON. Flags for non-standard features (comments, trailing commas) are ignored.
Workflow:
yyjson_incr_new().yyjson_incr_read() repeatedly with increasing byte lengths.err.code == YYJSON_READ_ERROR_MORE, continue reading.yyjson_incr_free().const char *dat = your_file.bytes;
size_t len = your_file.size;
yyjson_read_flag flg = YYJSON_READ_NOFLAG;
yyjson_incr_state *state = yyjson_incr_new(dat, len, flg, NULL);
yyjson_doc *doc;
yyjson_read_err err;
size_t read_so_far = 0;
do {
read_so_far += 100000; // Increment by several KB/MB for efficiency
if (read_so_far > len) read_so_far = len;
doc = yyjson_incr_read(state, read_so_far, &err);
if (err.code != YYJSON_READ_ERROR_MORE) break;
} while (read_so_far < len);
yyjson_incr_free(state);
if (doc != NULL) {
// ... use doc
yyjson_doc_free(doc);
}const char *dat = your_file.bytes;
size_t len = your_file.size;
yyjson_read_flag flg = YYJSON_READ_NOFLAG;
yyjson_incr_state *state = yyjson_incr_new(dat, len, flg, NULL);
yyjson_doc *doc;
yyjson_read_err err;
size_t read_so_far = 0;
do {
read_so_far += 100000;
if (read_so_far > len)
read_so_far = len;
doc = yyjson_incr_read(state, read_so_far, &err);
if (err.code != YYJSON_READ_ERROR_MORE)
break;
} while (read_so_far < len);
yyjson_incr_free(state);
if (doc != NULL) { ... }
yyjson_doc_free(doc);Use yyjson_mut_doc and its related APIs to construct JSON documents from scratch.
Important Considerations:
yyjson_mut_doc uses a memory pool for all strings and values. The pool can only be created, grown, or freed in its entirety. This makes the API more suitable for write-once operations rather than frequent mutation of existing documents.yyjson_mut_val can only be added to one object or array at a time. Attempting to add a value that is already part of another container will result in incorrect behavior.// Build this JSON:
// {
// "page": 123,
// "names": [ "Harry", "Ron", "Hermione" ]
// }
// Create a mutable document.
yyjson_mut_doc *doc = yyjson_mut_doc_new(NULL);
// Create an object, the value's memory is held by doc.
yyjson_mut_val *root = yyjson_mut_obj(doc);
// Create key and value, add to the root object.
yyjson_mut_val *key = yyjson_mut_str(doc, "page");
yyjson_mut_val *num = yyjson_mut_int(doc, 123);
yyjson_mut_obj_add(root, key, num);
// Create 3 string values, add to the array object.
yyjson_mut_val *names = yyjson_mut_arr(doc);
yyjson_mut_val *name1 = yyjson_mut_str(doc, "Harry");
yyjson_mut_val *name2 = yyjson_mut_str(doc, "Ron");
yyjson_mut_val *name3 = yyjson_mut_str(doc, "Hermione");
yyjson_mut_arr_append(names, name1);
yyjson_mut_arr_append(names, name2);
yyjson_mut_arr_append(names, name3);
yyjson_mut_obj_add(root, yyjson_mut_str(doc, "names"), names);
// ❌ Wrong! the value is already added to another container.
// yyjson_mut_obj_add(root, key, name1);
// Set the document's root value.
yyjson_mut_doc_set_root(doc, root);
// Write to JSON string
const char *json = yyjson_mut_write(doc, 0, NULL);
// Free the memory of doc and all values created from this doc.
yyjson_mut_doc_free(doc);To build the library from source using CMake, create a build directory and run the build commands. You can choose to build either a static or a shared library.
# Create build directory
git clone https://github.com/ibireme/yyjson.git
cmake -E make_directory build; cd build
# Build static library
cmake ..
cmake --build .
# Build shared library
cmake .. -DBUILD_SHARED_LIBS=ON
cmake --build .You can use the vcpkg dependency manager to install yyjson by following these steps:
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh # ./bootstrap-vcpkg.bat for PowerShell
./vcpkg integrate install
./vcpkg install yyjson