jzon

repository·develop·Indexed 41 days ago

https://github.com/zulu-inuoe/jzon

A correct and safe JSON reader/writer for Common Lisp following RFC 8259. It provides high-level functions like `jzon:parse` and `jzon:stringify`, a low-level `jzon:writer` API for manual construction, and a streaming parser via `jzon:make-parser` and `jzon:parse-next` for processing large files.

Tokens
6K
Snippets
21
Records
25
Agent score
3%

What's inside jzon

  1. Understand jzon type mappings

    develop

    jzon maps JSON types to Common Lisp types as follows:

    JSONCL
    truesymbol t
    falsesymbol nil
    nullsymbol null (Note: cl:null is used as a sentinel)
    numberinteger or double-float
    stringsimple-string
    arraysimple-vector
    objecthash-table (equal)
  2. Serialize standard-objects using slot types

    develop

    By default, jzon serializes a standard-object as a JSON object using its bound slots as keys. You can control how nil values are serialized by specifying a :type on the slot. This is useful for ensuring that empty or missing data conforms to a specific JSON schema.

    Nil resolution rules based on :type:

    1. boolean $\rightarrow$ false
    2. list $\rightarrow$ []
    3. null $\rightarrow$ null (this is also the default if no type is specified)

    Example: If a class has a slot married with :type boolean, a nil value will serialize to false. If a slot children has :type list, nil will serialize to [].

    (defclass person () 
      ((married :initarg :married :type boolean)
       (children :initarg :children :type list)
       (job     :initarg :job)))
    
    ;; Serializing a person with nil values:
    (jzon:stringify (make-instance 'person :name "Anya" :job nil :married nil :children nil)
                    :pretty t :stream t)
    
    ;; Resulting JSON:
    ;; {
    ;;   "name": "Anya",
    ;;   "job": null,
    ;;   "married": false,
    ;;   "children": []
    ;; }
  3. Leverage base-string coercion for reduced memory usage

    develop
    jzon attempts to coerce strings to cl:simple-base-string whenever possible. On implementations like SBCL, this significantly reduces memory overhead because base-string can be represented using 8 bits per character, whereas standard string types may be stored internally as UTF32.
  4. How symbol keys are serialized in jzon:stringify

    develop

    When a symbol is used as a key in a JSON object during serialization, jzon applies specific casing rules by default:

    1. Downcased: Symbols are downcased unless they contain mixed-case characters.
    2. Preserved: Symbols with mixed-case characters (e.g., |noChange|) retain their case.
    3. Strings: If the key is already a string, it is used as-is.

    Example behavior:

    • :only-keys becomes "only-keys"
    • |noChange| becomes "noChange"
    • "AS A" becomes "AS A"
    (let ((ht (make-hash-table :test 'equal)))
      (setf (gethash 'only-keys ht) 'are-affected)
      (setf (gethash '|noChange| ht) '|when used|)
      (setf (gethash "AS A" ht) '|value|)
    
      (jzon:stringify ht :pretty t :stream t))
    
    ;; Output:
    ;; {
    ;;   "only-keys": "ARE-AFFECTED",
    ;;   "noChange": "when Used",
    ;;   "AS A": "value"
    ;; }
  5. Understand jzon's performance and safety trade-offs

    develop

    jzon is designed to provide high performance (aiming for within 50% of jsown speeds) while maintaining strict safety and correctness. Unlike some high-performance libraries like jsown or jonathan, jzon avoids process crashes (like access violations or memory faults) when encountering malformed JSON and does not produce incorrect results for invalid inputs (e.g., it won't treat this as true or no as null).

    Use jzon when you need a balance of high-speed parsing and robust error handling that prevents your Lisp process from crashing on stray characters or invalid syntax.

  6. Optimize memory with object key pooling

    develop

    By default, jzon:parse and jzon:make-parser implement object key pooling. This means that if a nested JSON object contains duplicate keys, the resulting Lisp strings for those keys will be shared (eq rather than string=).

    This is highly efficient for JSON payloads with many repeated keys. You can change this behavior by providing a custom :key-fn to the parsing functions.

    ;; The string "x" is shared (eq) between all 3 objects due to key pooling
    (jzon:parse "[{\"x\": 5}, {\"x\": 10}, {\"x\": 15}]")
  7. Use the jzon:writer API for manual JSON construction

    develop

    The jzon:writer API provides low-level control over the values, order, and types written to a JSON stream. This is useful for complex or highly customized JSON generation.

    Core Workflow

    1. Initialize: Use jzon:make-writer to create a writer or jzon:with-writer* to manage a writer within a scope.
    2. Write Structure: Use functions like jzon:begin-array*, jzon:end-array*, jzon:begin-object*, or jzon:end-object* to manage nesting.
    3. Write Data: Use jzon:write-value* for atoms, or specialized functions like jzon:write-property* (key and value) and jzon:write-keys*.

    Helper Macros

    • jzon:with-array*: Opens a block for array elements and automatically closes it.
    • jzon:with-object*: Opens a block for object properties and automatically closes it.
    • jzon:write-array*: Writes an entire array in one call.
    • jzon:write-object*: Writes an entire object in one call.

    Note on Suffixes

    Functions suffixed with * (e.g., jzon:write-value*) use the special variable jzon:*writer* and do not require the writer as the first argument. Functions without the suffix (e.g., jzon:write-value) require the writer to be passed explicitly as the first argument.

    (jzon:with-writer* (:stream *standard-output* :pretty t)
      (jzon:with-object*
        (jzon:write-key* :age)
        (jzon:write-value* 24)
        
        (jzon:write-property* :colour :blue)
        
        (jzon:write-properties* :outside nil
                                :interests #()
                                :talent 'null)
    
        (jzon:write-key* "an-array")
        (jzon:with-array*
          (jzon:write-values* :these :are :elements))
    
        (jzon:write-key* "another array")
        (jzon:write-array* :or "you" "can use this")))
  8. Manage parser lifecycle with jzon:with-parser

    develop

    To ensure that parsers (especially those opening files) are properly closed and resources are released, use the jzon:with-parser macro. It behaves similarly to with-open-file by wrapping jzon:make-parser with unwind-protect and jzon:close-parser.

    (jzon:with-parser (parser "path/to/file.json")
      ;; Use parser here
      ...)
  9. Set up a package local nickname for jzon

    develop

    To simplify usage in your REPL, you can set up a package local nickname. Use the command corresponding to your Lisp implementation:

    For SBCL:

    (sb-ext:add-package-local-nickname '#:jzon '#:com.inuoe.jzon)

    For other implementations (using uiop):

    (uiop:add-package-local-nickname '#:jzon '#:com.inuoe.jzon)
    ;; SBCL
    (sb-ext:add-package-local-nickname '#:jzon '#:com.inuoe.jzon)
    
    ;; Other
    (uiop:add-package-local-nickname '#:jzon '#:com.inuoe.jzon)