repository·main·Indexed 42 days ago
https://github.com/juanjodiaz/streamparser-jsonA high-performance, spec-compliant streaming JSON parser for JavaScript, supporting Node.js, Deno, and the browser. It provides a core pure JavaScript implementation (@streamparser/json) with specialized wrappers for Node.js Transform streams (@streamparser/json-node) and WHATWG TransformStreams (@streamparser/json-whatwg). Version 0.0.26 allows for memory-efficient parsing of large JSON arrays and streams using JSONPath-like selectors and configurable buffering.
JSON.parse(...) for the data it processes.While JSON.parse() is the standard for synchronous JSON parsing, @streamparser/json provides several advantages for streaming and high-performance use cases:
JSON.parse() and works in the browser.JSONparse, which incorrectly allowed trailing commas or failed on certain character ranges).write operations (to be caught via try-catch), though an onError callback can be provided to prevent throwing.The parser participates in standard Node.js stream backpressure. If a downstream consumer is slow, the source is throttled.
Important Caveat: A single write() call is processed in one synchronous pass. This means all values contained within that one chunk are emitted at once. To ensure effective backpressure for large inputs, feed the parser in chunks (e.g., by piping from a file or socket) rather than passing a single large string or buffer in one write() call.
The throughput suite measures parsing performance in MB/s (decimal, 10⁶ B/s) by parsing representative JSON into materialized values. It compares the three public APIs—the core JSONParser, the Node Transform, and the WHATWG TransformStream—against native JSON.parse across four engines.
write operation representing the pure-parse upper bound (core + JSON.parse only).TransformStream a single multi-MB blob is not a valid test for streaming performance as it hits Node stream-queue pathologies.throughput/results.json. The suite uses adaptive warmup, median of N samples, and forced GC between samples. A CV% (Coefficient of Variation) greater than 15% is flagged with ! to indicate noisy data.JSON.parse, the suite focuses on the native-relative ratio as the portable performance metric rather than absolute MB/s.The parser integrates with standard WHATWG stream backpressure. If a downstream consumer is slow, the writer is throttled (its desiredSize drops), ensuring only a bounded number of parsed values are held in the readable buffer at a time. You can use standard pipeThrough or pipeTo methods to manage this.
Important Caveat: Backpressure applies between chunks, not within a single chunk. A single write() call is processed in one synchronous pass, meaning all values contained within that specific chunk are emitted and buffered immediately. To ensure effective backpressure for large inputs, feed the data in chunks (e.g., by piping from a fetch or file stream) rather than passing a single large document as one write() call.
The @streamparser ecosystem provides different flavors of the parser depending on your runtime and stream requirements:
@streamparser/json: Pure JavaScript implementation for parsing any JSON string or stream.@streamparser/json-whatwg: A wrapper around @streamparser/json that implements the WHATWG TransformStream interface.@streamparser/json-node: A wrapper around @streamparser/json that implements a Node.js Transform stream.Unlike JSONparse, which parses all valid objects in a stream, @streamparser/json is designed to stop after a single object unless a separator is explicitly configured.
end() method to stop processing. If the stream is in the middle of parsing an incomplete or incorrect JSON structure, end() will throw an error.onEnd callback to execute logic when the processing successfully finishes.end() ensures that any number currently being parsed is emitted before the process terminates.The @streamparser ecosystem provides different packages depending on how you want to consume the stream:
@streamparser/json: The core package. Use this to parse any JSON string or stream using pure JavaScript logic. This is the base implementation.@streamparser/json-whatwg: Use this if you are working in an environment that supports WHATWG Streams (like modern browsers or Deno) and want to wrap the parser into a TransformStream.@streamparser/json-node: Use this if you are working in Node.js and want to wrap the parser into a standard Node.js Transform stream.The @streamparser ecosystem provides different wrappers for the core parsing logic depending on your environment:
@streamparser/json: Pure JavaScript implementation for parsing any JSON string or stream.@streamparser/json-whatwg: Wraps the core logic into a WHATWG TransformStream (ideal for Browsers, Deno, and modern Node.js).@streamparser/json-node: Wraps the core logic into a Node.js Transform stream.The easiest way to use @streamparser/json-whatwg is via the JSONParser class, which acts as a WHATWG TransformStream. You can pipe a ReadableStream (like a file stream or a fetch response body) through the parser to receive parsed JSON values.
Values can be retrieved by piping to a destination stream or by manually using a reader to iterate through the stream.
import { JSONParser } from '@streamparser/json-whatwg';
const inputStream = new ReadableStream({
async start(controller) {
controller.enqueue('{ "test": ["a"] }');
controller.close();
},
});
const parser = new JSONParser();
// Option 1: Pipe to a destination
const reader = inputStream.pipeThrough(parser).getReader();
// Option 2: Manually reading values
while (true) {
const { done, value } = await reader.read();
if (done) break;
processValue(value);
// For the input above, values emitted: "a", ["a"], { test: ["a"] }
}To quickly parse a JSON stream in Node.js, import JSONParser from @streamparser/json-node. You can use it by piping an input stream into the parser and then piping the parser into a destination stream, or by listening to the data, error, and end events to process values as they are parsed.
import { JSONParser } from '@streamparser/json-node';
const parser = new JSONParser();
// Using pipes
inputStream.pipe(parser).pipe(destinationStream);
// Or using events
parser.on("data", (value) => { /* ... */ });
parser.on("error", err => { /* ... */ });
parser.on("end", () => { /* ... */ });If you need to replicate specific behaviors or constraints found in the older JSONparse library, you can use the following configuration options:
JSONparse's buffer strategy), set the stringBufferSize option to 64 * 1024.parseNumber method.end() method to complete the processing.// Example: Replicating JSONparse memory constraints
const parser = new StreamParser({
stringBufferSize: 64 * 1024
});