DOCSEARCHbyAlgolia
Submit Docs

RapidJSON

repository·master·Indexed 12 days ago

https://github.com/tencent/rapidjson

A high-performance, header-only C++ JSON parser and generator supporting both SAX and DOM-style APIs. Version 1.0.4 provides features such as SSE2/SSE4.2 acceleration, support for UTF-8, UTF-16, and UTF-32 encodings, and in-situ parsing to minimize memory overhead. It is fully compliant with RFC7159/ECMA-404 and offers optional support for relaxed syntax including comments and trailing commas.

Tokens
43.8K
Snippets
111
Records
180
Agent score
92%

What's inside RapidJSON

  1. Overview of RapidJSON APIs and Features

    master

    RapidJSON provides two primary styles of interaction with JSON data:

    • DOM API: Parses JSON into a tree-like structure (Document) that can be traversed and modified. Best for complex manipulations.
    • SAX API: A stream-based parser that triggers events during parsing. It is extremely fast and memory-efficient (the SAX parser is only ~500 lines of code).

    Key Features

    • Performance: Highly optimized; can be comparable to strlen(). Supports SSE2/SSE4.2 acceleration.
    • Memory Efficiency: Each Value occupies exactly 16 bytes on most 32/64-bit machines (excluding text strings).
    • Unicode Support: Supports UTF-8, UTF-16, and UTF-32 (LE & BE) with internal validation and transcoding.
    • Compliance: Fully compliant with RFC7159/ECMA-404, with optional support for relaxed syntax (comments, trailing commas, NaN/Infinity).
    • Header-only: No external dependencies like BOOST or even STL required.

  2. Overview of RapidJSON Stream Types

    master

    RapidJSON provides several stream abstractions to handle different data sources and encoding requirements:

    • Memory Streams: Simple streams for handling data already in memory.
    • File Streams: Optimized for reading from or writing to the file system, which can reduce memory consumption for large JSON files.
    • Encoded Streams: Used to convert between byte streams and character streams (e.g., handling different text encodings).
    • Custom Streams: Users can implement the IStreamWrapper or OStreamWrapper interfaces to integrate any custom input/output source with RapidJSON.

  3. Handling JSON strings with null characters

    master

    RapidJSON supports JSON strings containing the Unicode character U+0000 (escaped as "\u0000" in JSON). Because C/C++ strings are typically null-terminated, standard functions like strlen() will stop at the first \u0000.

    To correctly handle the full length of a string that may contain null characters, use GetStringLength() instead of strlen(). This is also more efficient as it avoids an extra pass over the string.

    // If JSON is { "s" : "a\u0000b" }
// GetStringLength() returns 3, while strlen() returns 1.
size_t len = document["s"].GetStringLength();
const char* str = document["s"].GetString();

  4. Query JSON Values by Type

    master

    Since a Value can hold different types, you should verify the type using IsXXX() methods before accessing the data with GetXXX() methods. RapidJSON does not perform automatic type conversion; calling a getter for the wrong type (e.g., GetInt() on a string) will cause an assertion failure in debug mode or undefined behavior in release mode.

    Common type checks and getters:

    • String: IsString() / GetString()
    • Boolean: IsBool() / GetBool()
    • Null: IsNull()
    • Number: IsNumber() (general check)
    • Integer: IsInt() / GetInt(), IsUint() / GetUint(), IsInt64() / GetInt64(), IsUint64() / GetUint64()
    • Floating Point: IsDouble() / GetDouble()
    • Array: IsArray()
    • Object: IsObject()
    assert(document.HasMember("hello"));
assert(document["hello"].IsString());
printf("hello = %s\n", document["hello"].GetString());

assert(document["i"].IsNumber());
assert(document["i"].IsInt());          
printf("i = %d\n", document["i"].GetInt());

  5. Resolve remote schemas using IRemoteSchemaDocumentProvider

    master

    JSON Schema supports the $ref keyword for referencing local or remote schemas via URIs. Since SchemaDocument cannot resolve URIs automatically, you must implement the IRemoteSchemaDocumentProvider interface to provide a mechanism for resolving these references.

    class MyRemoteSchemaDocumentProvider : public IRemoteSchemaDocumentProvider {
public:
    virtual const SchemaDocument* GetRemoteDocument(const char* uri, SizeType length) {
        // Resolve the uri and return a pointer to that schema.
    }
};

// ...
MyRemoteSchemaDocumentProvider provider;
SchemaDocument schema(sd, &provider);

  6. Use a DOM as a SAX event publisher

    master

    In RapidJSON, the Value::Accept() method is used to publish SAX events related to a value to a handler. This decouples the data (Value) from the processing logic (Writer or custom handlers).

    While Writer is commonly used to generate JSON from a DOM, you can implement a custom handler to transform a DOM into other formats, such as XML.

    // Using a Writer to generate JSON from a DOM
Writer<StringBuffer> writer(buffer);
d.Accept(writer);

  7. Understand the JSON Schema Test Suite structure

    master

    The JSON Schema Test Suite is a language-agnostic collection of JSON objects designed to test JSON Schema validation libraries. It is organized by schema draft/version (e.g., draft3) within the tests directory.

    Each .json file in a draft directory contains an array of test case objects. To implement these tests in your own framework, you must parse these JSON files and map the schema and test data to your validator's API. Each test case object follows this structure:

    • description: A string describing the test case.
    • schema: The JSON schema object to validate against.
    • tests: An array of specific test instances, where each instance contains:
      • description: A string describing the specific test.
      • data: The JSON instance to be validated.
      • valid: A boolean indicating whether the instance is expected to be valid (true) or invalid (false).
    {
    "description": "the description of the test case",
    "schema": {"the schema that should" : "be validated against"},
    "tests": [
        {
            "description": "a specific test of a valid instance",
            "data": "the instance",
            "valid": true
        },
        {
            "description": "another specific test this time, invalid",
            "data": 15,
            "valid": false
        }
    ]
}

  8. How SAX-style parsing and generation work in RapidJSON

    master

    RapidJSON provides a SAX (Simple API for XML) style interface for high-performance JSON processing.

    • Reader (or GenericReader): A SAX-style parser that reads JSON from a stream and publishes events (like StartObject, Key, String, etc.) to a user-defined Handler.
    • Writer (or GenericWriter): A SAX-style generator that converts events into a JSON string.

    This approach is often faster than the DOM-style (Document) approach because it avoids building an in-memory tree structure, instead processing tokens one by one as they are encountered.

    struct MyHandler : public BaseReaderHandler<UTF8<>, MyHandler> {
    bool Null() { return true; }
    bool Bool(bool b) { return true; }
    bool Int(int i) { return true; }
    // ... other event handlers
};

MyHandler handler;
Reader reader;
StringStream ss(json);
reader.Parse(ss, handler);

  9. Use AutoUTF for runtime encoding detection

    master
    If the encoding of your input or output stream is not known until runtime, use AutoUTF. This encoding is designed to choose the appropriate encoding based on the stream content. AutoUTF should be used in conjunction with EncodedInputStream and EncodedOutputStream.

  10. Understand RapidJSON move semantics

    master

    RapidJSON uses move semantics for assignment and modifying functions (like AddMember() and PushBack()) to maximize performance. When you assign one Value to another, the source value is moved to the destination, and the source becomes a Null value. This avoids expensive deep copies of variable-sized types like String, Array, and Object.

    Key behavior:

    • a = b; results in a taking b's content, and b becoming Null.
    • This applies to AddMember() and PushBack() as well.
    • For C++03 compatibility, RapidJSON implements this via assignment operators that transfer ownership.
    Value a(123);
Value b(456);
a = b;         // b becomes a Null value, a becomes number 456.

  11. Implement a custom RapidJSON Stream

    master

    You can create custom stream classes (e.g., for network sockets or compressed files) by implementing the Stream concept.

    For Input Streams, implement:

    • Ch Peek() const: Returns the current character without moving the cursor.
    • Ch Take(): Returns the current character and moves the cursor.
    • size_t Tell(): Returns the number of characters read.

    For Output Streams, implement:

    • void Put(Ch c): Writes a character.
    • void Flush(): Flushes the buffer.

    For In-Situ Parsing (Optional): If you need to support in-situ parsing, you must also provide implementations for:

    • Ch* PutBegin()
    • size_t PutEnd(Ch* begin)

    Note: Even if you don't implement the in-situ methods, you must provide empty implementations to avoid compilation errors.

    concept Stream {
    typename Ch;    //!< Stream's character type

    //! Read current char without moving cursor
    Ch Peek() const;

    //! Read current char and move cursor
    Ch Take();

    //! Get current position
    size_t Tell();

    //! For in-situ parsing: return pointer to start of write
    Ch* PutBegin();

    //! Write a character
    void Put(Ch c);

    //! Flush buffer
    void Flush();

    //! For in-situ parsing: complete write operation
    size_t PutEnd(Ch* begin);
}

  12. Core utility classes: Allocator, Encoding, and Stream

    master

    Both SAX and DOM APIs rely on three fundamental utility concepts:

    1. Allocator: Manages memory allocation.
    2. Encoding: Handles character encoding (e.g., UTF-8).
    3. Stream: Provides the interface for reading from or writing to data sources.