jsonic

While the Go version is a port of the TypeScript implementation, there are key behavioral differences to note when parsing:

Number + Text Tokenization

In TypeScript, an input like 123abc is split into a number (123) and text (abc). In Go, this is treated as a single text token ("123abc") and is not recognized as a number.

Empty and Whitespace Input

• Exact empty string (""): Both implementations short-circuit immediately.
• Whitespace or comments only: Both implementations process these through the normal parse flow, which resolves to null (TypeScript) or nil (Go) based on grammar rules.

Jsonic is primarily an LL(2) parser. This means its lookahead capabilities are limited to a maximum of two tokens.

• Matching: The s: (selector) operator can only match s0 and s1 (the current and next tokens).
• Conditions: The c: (condition) operator can inspect ctx.t0 and ctx.t1.
• Constraint: Jsonic does not lex tokens 3+ ahead eagerly. If a production's decisive prefix requires more than two tokens to distinguish it from other rules, you must split that production into auxiliary rules, creating one rule for each decision point.

Attempting to use lookahead beyond two tokens will fail because the data has not yet been computed by the lexer.

Jsonic is not a classical Context-Free Grammar (CFG) engine; it is a rule-driven push-down state machine that uses an explicit two-token lookahead.

A grammar is composed of rules, where each rule has two states: Open (o) and Close (c). Each state contains an ordered list of AltSpecs (alternates). The parser attempts each alternate in order, and the first match wins.

Key components of a grammar include:

• Rules: Defined by their open and close states.
• Tokens: First-class entities including fixed tokens, regex matchers, and token sets (e.g., IGNORE, VAL, KEY).
• State Hooks: bo (before open), ao (after open), bc (before close), and ac (after close) are used to assemble the resulting rule.node.

jsonic is a lenient parser that accepts standard JSON plus several relaxations to make data entry easier. Key features include:

• Unquoted keys: a:1 becomes {"a": 1}
• Implicit objects: a:1,b:2 becomes {"a": 1, "b": 2}
• Implicit arrays: a,b,c becomes ["a", "b", "c"]
• Comments: Supports #, //, and /* */
• Single/backtick quotes: Supports 'hello' and `hello`
• Path diving: a:b:1 becomes {"a": {"b": 1}}
• Trailing commas: {a:1,} is valid
• All number formats: Supports hex, octal, binary, and separators

The jsonic parser is designed as a stack-based rule engine with 2-token lookahead. While the current implementation uses a 'fail-fast' model (throwing a JsonicError on the first error), it is architecturally feasible to implement Language Server Protocol (LSP) support by extending the parser to collect multiple errors and resume parsing at 'sync points'.

To support LSP-style diagnostics, the parser must transition from a single-shot error model to a multi-error collection model. This involves:

1. Accumulating errors in a ctx.errs array within the Context.
2. Using a dynamic sync-point mechanism to skip tokens until a safe state is reached.
3. Making this behavior opt-in via configuration to avoid breaking existing consumers who rely on the 'first error throws' behavior.

The Go implementation of jsonic treats all numbers as float64.

Note on number+text edge cases: Unlike the TypeScript version, input containing a number followed by text (e.g., 123abc) is rejected as a number and parsed as text in Go. It does not produce separate number and text tokens.

Jsonic does not have direct equivalents for Yacc-style precedence declarations like %left, %right, or %nonassoc. To implement precedence in a grammar, you must use one of the following strategies:

1. Grammar Stratification: Rewrite the grammar into a hierarchy of rules (e.g., expr → term → factor) to enforce order of operations.
2. Precedence Climbing: Use n: counters to track precedence levels and c: conditions to guard alternates. For example, you can guard an alternate with a condition like c: { 'n.prec': { $lte: X } } to ensure it only matches when the current precedence is less than or equal to X.

jsonic extends standard JSON arrays with the following features:

• Implicit Top-Level Array: If the input contains comma- or newline-separated values without colons, it is parsed as an array.
• Named Properties in Arrays: By default, key-value pairs are allowed inside arrays (e.g., [a:1, b:2] becomes [{"a": 1}, {"b": 2}]). This behavior can be toggled using the list.property option.

// Implicit Top-Level Array
a, b, c               // → ["a", "b", "c"]
1, 2, 3               // → [1, 2, 3]

// Named Properties in Arrays
[a:1, b:2]            // → [{"a": 1}, {"b": 2}]

You can enable options to receive richer metadata about the parsed JSON structure instead of standard Go types. This is useful for preserving formatting information like quote characters or detecting implicit syntax (missing brackets/braces).

• Set TextInfo: true to receive jsonic.Text instead of string. This preserves the Quote (rune) used.
• Set ListRef: true to receive jsonic.ListRef instead of []any. This includes an Implicit flag indicating if brackets were omitted.
• Set MapRef: true to receive jsonic.MapRef instead of map[string]any. This includes an Implicit flag indicating if braces were omitted.

// Example of extended types (conceptual usage)
// TextInfo: true -> jsonic.Text{Quote rune, Str string}
// ListRef: true -> jsonic.ListRef{Val []any, Implicit bool, ...}
// MapRef: true  -> jsonic.MapRef{Val map[string]any, Implicit bool}

jsonic provides specific behaviors for structural characters:

• Trailing Commas: Always allowed and ignored in both objects and arrays.
• Auto-Close: Unclosed { or [ at the end of the input are automatically closed. This can be disabled using the rule.finish option.

// Trailing Commas
{a:1, b:2,}           // → {"a": 1, "b": 2}
[1, 2, 3,]            // → [1, 2, 3]

// Auto-Close
{a:1                  // → {"a": 1}
[1, 2                 // → [1, 2]

While the Jsonic parser is natively designed as a fail-fast system, its architecture is compatible with a 'panic-mode' error recovery scheme required for Language Server Protocol (LSP) support.

Key architectural elements that enable recovery include:

• Stack-based rules
• Group-tagged alternates
• Non-throwing lexer
• Iterative main loop

The proposed design for recovery involves adding an option subtree and a helper function to derive synchronization behavior from the existing grammar information, without changing the default fail-fast behavior of the API.

jsonic supports multiple string delimiters and unquoted strings:

• Quoted Strings: Supports double quotes ("), single quotes ('), and backticks (`).
• Unquoted Strings: Values that are not numbers, booleans, or null are treated as unquoted strings. They extend to the next structural character (comma, colon, bracket, brace) or the end of the line.
• Multiline Strings: Backtick strings (`) can span multiple lines; newlines are preserved.
• Indent-Adjusted Strings: Triple-quoted strings ('''...''') automatically strip common leading indentation.

// Quoted Strings
"hello"   'hello'   `hello`

// Unquoted Strings
{a: hello world}      // → {"a": "hello world"}

// Multiline Strings
`line one
line two`             // → "line one\nline two"

// Indent-Adjusted Strings
  '''
  line one
  line two
  '''                 // → "line one\nline two"