repository·master·Indexed 25 days ago
https://github.com/valyala/fastjsonA high-performance JSON parser and validator for Go designed to be faster than the standard encoding/json package by avoiding reflection and schema requirements. It allows efficient access to multiple fields by parsing input only once. The library includes a Parser for efficient traversal, an Arena for fast creation and reuse of Value objects, and handy helper functions for quick one-off extractions.
When using fastjson, be aware of the following security and memory considerations:
fastjson can require up to sizeof(Value) * len(inputJSON) bytes of memory. To prevent exhaustion attacks, always limit the maximum size of the inputJSON byte slice before passing it to the parser.fastjson is designed not to crash or panic on specially crafted malicious input; it will return an error for invalid JSON instead.fastjson is that references to objects returned by a Parser or an Arena must be released before the next call to Parse or the next use of that Arena. If you hold onto references to objects from a previous parse while performing a new parse with the same Parser instance, the program may behave improperly.To run fuzzing on fastjson, install go-fuzz and its build tools, then use go-fuzz-build and go-fuzz with a corpus:
# Install tools
go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
# Run fuzzing
mkdir -p workdir/corpus
cp $GOPATH/src/github.com/dvyukov/go-fuzz-corpus/json/corpus/* workdir/corpus
go-fuzz-build github.com/valyala/fastjson
go-fuzz -bin=fastjson-fuzz.zip -workdir=workdirTo achieve maximum performance with fastjson, follow these patterns:
fastjson.Parser and fastjson.Scanner instances for multiple JSONs to reduce memory allocation overhead. Consider using fastjson.ParserPool for concurrent environments.fastjson.Get* multiple times (which re-parses the input each time), use p.Parse() once and call Value.Get* on the resulting Value.Value.Get once for the common prefix, then call Value.Get* on the returned sub-value for the specific suffixes.Value.GetArray instead of calling Value.Get* for every individual index.An Arena is used for the fast creation and reuse of Value objects. It manages a buffer to minimize allocations during JSON construction.
Typical Arena lifecycle:
Arena and its New* methods (e.g., NewObject, NewString) to build your JSON structure.Value tree into JSON bytes using Value.MarshalTo.Arena.Reset() to clear all allocated values at once.Arena instance.Warning: Calling Arena methods from concurrent goroutines is unsafe. For concurrent usage, use per-goroutine Arena instances or an ArenaPool.
// Typical Arena lifecycle:
//
// 1. Construct Values via the Arena and Value.Set* calls.
// 2. Marshal the constructed Values with Value.MarshalTo call.
// 3. Reset all the constructed Values at once by Arena.Reset call.
// 4. Go to 1 and re-use the Arena.The Scanner is used to parse a series of JSON values from a single input string or byte slice. It is particularly useful for parsing JSON Lines (JSONL) or streams where multiple JSON objects are delimited by whitespace.
Key characteristics:
Scanner can be re-used for subsequent parsing by calling Init or InitBytes again.Scanner is not thread-safe and cannot be used from concurrent goroutines.Next(), the parsed value is accessible via Value(). This value is only valid until the next call to Next().If your program crashes while using fastjson, it is likely due to improper usage of the parser or scanner. Check for the following common issues:
Parser or Scanner beyond the next call to Parser.Parse or Scanner.Next if the documentation specifies such a restriction.fastjson objects from multiple goroutines simultaneously if the documentation indicates they are not thread-safe.-race flag to detect data races. Ensure the race detector reports zero races.If these steps do not resolve the crashes, file a bug report on GitHub.
For simple tasks where you only need to extract one specific value, use the fastjson.GetInt (or similar Get* functions) one-liner. This is convenient but note that each call re-parses the entire input JSON, so it is inefficient for multiple field accesses.
s := []byte(`{"foo": [123, "bar"]}`)
fmt.Printf("foo.0=%d\n", fastjson.GetInt(s, "foo", "0"))
// Output:
// foo.0=123To efficiently access multiple fields, create a fastjson.Parser instance, call Parse to get a Value, and then use the Get* methods on that Value. This approach parses the input only once.
var p fastjson.Parser
v, err := p.Parse(`{
"str": "bar",
"int": 123,
"float": 1.23,
"bool": true,
"arr": [1, "foo", {}]
}`)
if err != nil {
log.Fatal(err)
}
fmt.Printf("foo=%s\n", v.GetStringBytes("str"))
fmt.Printf("int=%d\n", v.GetInt("int"))
fmt.Printf("float=%f\n", v.GetFloat64("float"))
fmt.Printf("bool=%v\n", v.GetBool("bool"))
fmt.Printf("arr.1=%s\n", v.GetStringBytes("arr", "1"))
// Output:
// foo=bar
// int=123
// float=1.230000
// bool=true
// arr.1=foofastjson is primarily a parser, it provides a way to perform marshaling (serialization) via the Value.MarshalTo method. For extremely high-performance JSON marshaling requirements, the author recommends using quicktemplate instead.ValidateBytes(b []byte) to check if a byte slice contains valid JSON. This is a convenience wrapper around Validate for raw byte data.To prepare a Scanner for parsing, use either Init (for strings) or InitBytes (for byte slices). The input may contain multiple JSON values delimited by whitespace.
Init(s string): Initializes the scanner with the provided string.InitBytes(b []byte): Initializes the scanner with the provided byte slice.