simdjson

If you are compiling with C++20 or later, you can integrate the simdjson On-Demand API with std::ranges and range adaptors (like std::views::transform) using helper functions. These wrappers are zero-cost and forward directly to the underlying On-Demand iterators without buffering values.

Array Iteration: Use ondemand::get_range() on an ondemand::array. This produces a std::ranges::view that satisfies std::ranges::input_range.

Object Iteration: Use ondemand::get_key_value_range() on an ondemand::object. This yields simdjson_result<ondemand::field> elements, allowing you to access both the key and the value.

Error Handling Compatibility: These helpers work with both exception-based and non-exception-based code patterns.

#include "simdjson.h"
#include <ranges>
#include <string>
#include <vector>

auto json = R"([
  { "name": "Alice", "age": 30 },
  { "name": "Bob",   "age": 25 },
  { "name": "Carol", "age": 35 }
])"_padded;

ondemand::parser parser;
auto doc = parser.iterate(json);
auto arr = doc.get_array();

// Use std::views::transform to extract names
auto names = ondemand::get_range(arr)
  | std::views::transform([](auto elem) -> std::string {
      return std::string(std::string_view(elem["name"]));
    });

for (auto name : names) {
  std::cout << name << std::endl;  // Alice, Bob, Carol
}

Choosing between the On-Demand API and the core simdjson API depends on your hardware constraints, data reliability, and performance requirements.

Use the On-Demand API when:

• Hardware is known at compile time: You are targeting specific 64-bit hardware (especially on x64). Note that on 64-bit ARM, runtime dispatching is generally unnecessary.
• Data is pre-vetted or controlled: You are working with large, well-formed JSON files from a known dialect (e.g., static data dumps for machine learning) or you control both the JSON producer and consumer.
• Validation is not required: The specific parts of the JSON you are accessing do not need strict validation, and the layout follows a consistent schema.
• API ergonomics are a priority: You prefer a clean, flexible, and maintainable API over the strictness of a DOM-style parser.

Use the Core simdjson API when:

• Runtime dispatching is required: You need a single binary to run efficiently across different x64 processors without knowing the hardware at compile time.
• Strict validation is necessary: You are consuming JSON from external/untrusted systems and need to ensure the input is fully validated.
• Full document navigation is needed: You need to navigate through the entire document structure at will rather than accessing specific nodes on demand.

If your system supports C++20, the recommended way to deserialize JSON into custom types is by implementing a tag_invoke function within the simdjson namespace. This approach leverages C++20 concepts to allow automatic casting of generic JSON types (like double or int) to your specific types (like float or int).

To implement this, define a function with the following signature inside the simdjson namespace:

auto tag_invoke(deserialize_tag, simdjson_value &val, YourType& target)

Arguments:

• simdjson::deserialize_tag: The tag for the Customization Point Object (CPO).
• val: A simdjson value type (e.g., document, value, or document_reference).
• target: An instance of your custom type to be populated.

This allows you to use the type directly in constructors or with get<T>().

struct Car {
  std::string make;
  std::string model;
  int year;
  std::vector<float> tire_pressure;
};

namespace simdjson {
// This tag_invoke MUST be inside simdjson namespace
template <typename simdjson_value>
auto tag_invoke(deserialize_tag, simdjson_value &val, Car& car) {
  ondemand::object obj;
  auto error = val.get_object().get(obj);
  if (error) return error;

  if ((error = obj["make"].get_string(car.make))) return error;
  if ((error = obj["model"].get_string(car.model))) return error;
  if ((error = obj["year"].get(car.year))) return error;
  if ((error = obj["tire_pressure"].get<std::vector<float>>().get(car.tire_pressure))) return error;

  return simdjson::SUCCESS;
}
}

When requesting strings via .get_string(), simdjson returns a std::string_view. This avoids the overhead of allocating new std::string objects.

CRITICAL: The lifecycle of these std::string_view instances is tied to the ondemand::parser instance. If the parser is destroyed or reused for a new document, any existing std::string_view pointing to its buffers becomes invalid.

For object keys, you can use:

• key().raw(): Provides direct access to the unescaped string for fast ASCII comparisons (e.g., key() == "test"). The comparison is byte-by-byte and does not handle escaped characters for speed.
• unescaped_key(): Returns a std::string_view of the unescaped key.

Note: Once unescaped_key() is called on a field, you cannot call key() or unescaped_key() again on that same field instance; the key is consumed.

auto doc = parser.iterate(json);
for(auto field : doc.get_object())  {
  std::string_view keyv = field.unescaped_key();
}

The simdjson API can be used with or without exceptions. For more control or easier debugging, you can use the exception-free approach where all APIs that can fail return a simdjson_result<T>, which is a <value, error_code> pair.

Key Error Handling Rules:

• Check for success: Use .get() to retrieve the value. If the result is not simdjson::SUCCESS, an error occurred.
• Boolean evaluation: simdjson::SUCCESS evaluates to false in a boolean context. Any error code evaluates to true.
• Recoverable Errors:
  • simdjson::INCORRECT_TYPE: Occurs when converting a value to an unexpected type (e.g., treating an array as a number).
  • simdjson::NO_SUCH_FIELD: Occurs when querying a key that does not exist in an object.
• Fatal Errors:
  • simdjson::INCOMPLETE_ARRAY_OR_OBJECT and simdjson::TAPE_ERROR indicate the document is not valid JSON.
  • After a fatal error, doc.is_alive() returns false. Do not continue using the document instance after a fatal error.

Error Retrieval

You can retrieve a human-readable error message using simdjson::error_message(error).

ondemand::document doc;
auto error = parser.iterate(json).get(doc);
if(error) { std::cerr << simdjson::error_message(error); exit(1); }

The parse_many interface is designed for processing files or streams containing multiple small JSON documents (e.g., NDJSON, JSONL, or RFC 7464) efficiently. It allows you to process gigabytes of data using a small, fixed amount of memory by iterating through a document_stream.

Key Features:

• Incremental Processing: You don't need to load the entire file into memory; you can process documents one by one.
• Memory Efficiency: It uses a recycled parser object and a configurable batch_size to minimize allocations.
• Parallelism: If SIMDJSON_THREADS_ENABLED is active, the library uses a worker thread to perform Stage 1 parsing on the next batch while the main thread performs Stage 2 parsing on the current batch, significantly reducing overhead.

// Note: Actual C++ code implementation depends on the specific API signatures in basics.md
// but the conceptual usage pattern is:
// simdjson::document_stream stream = parser.parse_many(input_data, batch_size);
// for (auto doc : stream) {
//     // process doc
// }

To achieve maximum performance with simdjson::ondemand, follow these two access patterns:

1. Avoid redundant key lookups: Instead of repeatedly accessing a nested object via its parent key (e.g., o["data"]["key"]), capture the nested object once and then access its members. This reduces the number of times the parser must search for the key.
2. Access keys in document order: You will get better performance if you seek keys in the same order they appear in the JSON document. If the document is {"a":1, "b":2}, access "a" then "b". If the order is unknown, consider using key selectors which extract a fixed set of fields in a single pass regardless of order.

Note: For high-performance key matching, you can use a switch on the first character of the key to quickly narrow down potential matches before performing a full string comparison.

// AVOID THIS (Repeated lookups):
std::string_view make = o["data"]["make"];
std::string_view model = o["data"]["model"];
std::string_view year = o["data"]["year"];

// DO THIS (Capture object once):
simdjson::ondemand::object data = o["data"];
std::string_view model = data["model"];
std::string_view year = data["year"];
std::string_view rating = data["rating"];