DOCSEARCHbyAlgolia
Submit Docs

Edamame Clojure and EDN Parser

repository·master·Indexed 41 days ago

https://github.com/borkdude/edamame

A highly configurable EDN and Clojure parser that provides location metadata (row, column, end-row, end-column) for parsed elements. Designed for feedback tools, static analysis, and parsing Clojure-like expressions without evaluation. Features include support for reader conditionals, auto-resolving keywords, syntax quoting, and optimized JVM performance. Built-in for Babashka.

Tokens
4.2K
Snippets
17
Records
24
Agent score
53%

What's inside Edamame

  1. Performance improvements in JVM parsing

    master

    Edamame includes significant performance optimizations for JVM users, making it up to 2.26x faster than standard sci options and 1.25x faster than clojure.core/read when parsing large files like core.clj.

    Key optimizations include:

    • IndexingStringReader: A JVM-only deftype that replaces the standard tools.reader stack. It uses .charAt and primitive mutable fields for line/column bookkeeping to avoid the overhead of generic RT.nthFrom calls and multiple reader layers.
    • Fast Paths: Optimized implementations for readToken, readString, and skipWhitespace that use substrings and primitive char loops instead of StringBuilder or per-character protocol calls.
    • Reduced Allocation: The parser now uses integer locals instead of allocating a Loc record per form, and uses with-meta with direct associations instead of vary-meta with closures.
    • Optimized Context: Hot keys in the Options record are now declared as fields rather than living in an __extmap hash map, reducing record copying and map allocations.

  2. Auto-resolve Keywords

    master

    Edamame can automatically resolve aliased keywords (like ::str/foo) based on a provided mapping or the current namespace context.

    Using a mapping

    Pass a map to :auto-resolve where keys are aliases and values are namespaces.

    (parse-string "[::foo ::str/foo]" {:auto-resolve '{:current user str clojure.string}})
;;=> [:user/foo :clojure.string/foo]

    Using :auto-resolve-ns

    To automatically resolve keywords based on ns forms encountered in the string, set :auto-resolve-ns true. This is useful when parsing files that contain (ns ...) declarations.

    (parse-string-all "(ns foo (:require [clojure.set :as set])) ::set/foo" {:auto-resolve-ns true})
;;=> [(ns foo (:require [clojure.set :as set])) :clojure.set/foo]
    (parse-string "[::foo ::str/foo]" {:auto-resolve '{:current user str clojure.string}})
;;=> [:user/foo :clojure.string/foo]

(parse-string-all "(ns foo (:require [clojure.set :as set])) ::set/foo" {:auto-resolve-ns true})
;;=> [(ns foo (:require [clojure.set :as set])) :clojure.set/foo]

  3. Access Location Metadata

    master

    Edamame can attach location information (row, column, end-row, end-column) to parsed elements as metadata. You can control which elements receive this metadata using the :location? option.

    By default, locations are attached to elements that satisfy the predicate provided in :location? (the Clojure default is seq?).

    (def s "
[{:a 1}
 {:b 2}]")
(map meta (parse-string s))
;;=>
({:row 2, :col 2, :end-row 2, :end-col 8}
 {:row 3, :col 2, :end-row 3, :end-col 8})

  4. Understand ClojureDart (cljd) platform semantics in Edamame

    master

    When using Edamame on the ClojureDart platform, be aware of the following semantic differences compared to standard Clojure/ClojureScript:

    Numeric Types

    • No Ratio or BigDecimal: Values like 1/2 or 3.14M are parsed as double (similar to ClojureScript).
    • BigInt: The N suffix (e.g., 123N) parses as a Dart BigInt.

    String Normalization

    • Line endings \r\n and \r are normalized to \n during reading, matching tools.reader behavior.

    Syntax Quote

    • ns is treated as a special symbol (unqualified), matching ClojureScript and cljd.reader. The special set includes ns plus catch, finally, and &.

    Collections and Maps

    • No PersistentArrayMap: All maps are hash maps. Small map literals may lose insertion order. If insertion order is required, pass an ordered map constructor via the :map option.
    • List Meta Leak: Due to a ClojureDart bug, cljd.core/list (which replaces cljd.core/list in the parser via :refer-clojure :exclude) may leak const-folded compiler metadata into built lists.

  5. Install Edamame

    master

    Edamame can be used as a dependency in your Clojure project. If you are using Babashka, Edamame is built-in and does not require an additional dependency.

    ;; Add edamame as a dependency in your project (e.g., in deps.edn or project.clj)

  6. Run tests on the ClojureDart (cljd) branch

    master

    To run the Edamame test suite specifically for the ClojureDart port, use the following command. This requires the Dart SDK (>= 3.0) and the :cljd alias which pins ClojureDart to a specific git SHA.

    clj -M:cljd test edamame.core-test

    If you are developing on the ClojureDart codebase itself and want to use your local checkout, use the :cljd-local alias:

    clj -M:cljd:cljd-local test edamame.core-test
    clj -M:cljd test edamame.core-test

  8. Fix Incomplete Expressions

    master

    When Edamame encounters unmatched delimiters, it throws a clojure.lang.ExceptionInfo. You can use the ex-data from this exception to find the :edamame/expected-delimiter and automatically fix the expression by appending it.

    (defn fix-expression [expr]
  (try (when (parse-string expr) expr)
       (catch clojure.lang.ExceptionInfo e
         (if-let [expected-delimiter (:edamame/expected-delimiter (ex-data e))]
           (fix-expression (str expr expected-delimiter))
           (throw e)))))

(fix-expression "{:a (let [x 5") ;; => "{:a (let [x 5])}"
    (def incomplete "{:a (let [x 5")

(defn fix-expression [expr]
  (try (when (parse-string expr) 
         expr)
       (catch clojure.lang.ExceptionInfo e
         (if-let [expected-delimiter (:edamame/expected-delimiter (ex-data e))]
           (fix-expression (str expr expected-delimiter))
           (throw e)))))

(fix-expression incomplete) ;; => "{:a (let [x 5])}"

  9. Handle exceptions in ClojureDart (cljd)

    master

    In the ClojureDart port, cljd.core/ExceptionInfo is used because cljd ex-info is not a native Dart Exception.

    When writing assertions to check if an error was thrown, standard thrown? assertions might not work as expected if they are looking for a Dart Exception. Ensure your test logic accounts for cljd.core/ExceptionInfo.

  10. Prepare readers for parsing

    master

    To use parse-next, you must first coerce your input (string or java.io.Reader) into an Edamame reader:

    • reader: Coerces input into an indexing pushback-reader.
    • source-reader: Coerces input into a source-logging-reader (useful for tracking source information).
    (parse-next (reader "(a) (b)") (normalize-opts {:all true}))

  11. Parse forms from a reader with `parse-next`

    master

    Use parse-next to extract the next form from a reader. Note that opts must be passed through normalize-opts before being used with parse-next.

    parse-next+string is a variant that returns a vector containing the parsed value and the whitespace-trimmed string that was read.

    (parse-next (reader "(a) (b)") (normalize-opts {:all true}))

  12. Parse namespace forms with `parse-ns-form`

    master

    Use parse-ns-form to parse an s-expression representing a namespace definition into a map containing:

    • :name: The name of the namespace.
    • :aliases: A map of aliases to library names.
    (parse-ns-form '(ns my.ns :alias my-lib))
;;=> {:name my.ns, :aliases {my-lib my.lib}}