rewrite-clj

repository·main·Indexed 20 days ago

https://github.com/clj-commons/rewrite-clj

A library for reading and writing Clojure, ClojureScript, and EDN that preserves whitespace and comments to enable robust code manipulation and refactoring. It provides a Parser API for creating nodes, a Node API for inspection and analysis, and a Zip API for zipper-based navigation and editing of source code.

Tokens
8.7K
Snippets
27
Records
41
Agent score
68%

What's inside rewrite-clj

  1. Overview of rewrite-clj

    main
    rewrite-clj is a library designed to read and write Clojure, ClojureScript, and EDN (Extensible Data Notation) from within Clojure and ClojureScript. Its primary strength is its ability to perform these operations while preserving whitespace and comments, making it ideal for code refactoring, formatting, and automated transformations.
  2. Use positional support in rewrite-clj

    main

    Rewrite-clj v1 uses the positional support from rewrite-clj v0, which tracks row/column information even after zipper modifications.

    Key details:

    • Position Format: Positions are expressed as a [row-number col-number] vector.
    • ClojureScript Requirement: When using ClojureScript, you must explicitly enable positional support when creating the zipper by passing {:track-position? true} in the options map.
    • Compatibility: To maintain compatibility, rewrite-clj v1 supports both the v0 vector notation and the rewrite-cljs map notation ({:row r :col c}) for function parameters, but uses vector notation for function returns.
    ;; ClojureScript example: Enabling positional tracking
    (require '[rewrite-clj.zip :as z])
    (def zipper (z/of-string "[1 2 3]" {:track-position? true}))
  3. Customizing node skipping behavior in rewrite-clj

    main

    The design for custom node skipping allows users to define which nodes the zipper should skip during navigation (e.g., left, right, up, down, next, prev). While rewrite-clj currently hardcodes skipping for whitespace and comments, the proposed mechanism allows for a skip-node? predicate to be passed as an option during zipper creation.

    This predicate is used to determine if a node (or its ancestors) should be treated as 'invisible' during navigation. This is useful for skipping:

    • Reader discards (unevals): Nodes where (z/sexpr-able? zloc) is false.
    • (comment ...) forms: Entire lists that start with the comment symbol.
    • Custom criteria: Any other user-defined logic for node selection.
    ;; Concept: A skip-node? predicate would be passed during zipper creation.
    ;; It accepts a single argument: a zipper location (zloc).
    
    (defn my-skip-predicate [zloc]
      ;; Return true if the node should be skipped during navigation
      ...)
  4. Understand ClojureScript namespace workarounds

    main

    Due to Google Closure namespace handling in ClojureScript, some namespaces that work in Clojure clash in ClojureScript. To maintain compatibility, rewrite-clj v1 preserves specific naming conventions for ClojureScript internal namespaces.

    For example, while Clojure uses rewrite-clj.zip.find, ClojureScript uses rewrite-clj.zip.findz to avoid collisions.

  5. Generate code using import-vars templates

    main

    In rewrite-clj v1, import-vars functionality is handled via code generation from templates rather than runtime loading. This avoids the maintenance and stability issues of the potemkin library.

    Template Syntax

    Instead of the old import-vars macro, use a metadata map #_{:import-vars/import ...} in your template file (e.g., .cljc or .clj).

    Example Template Syntax:

    #_{:import-vars/import
       {:from [[my.ns1 ^{:deprecated "1.2.3"} obsolete-fn
                ^{:added "1.2.4"} new-fn]]}}

    Workflow

    1. Generate code: Run the generator to create target source files from templates.
    2. Review: You must manually review the generated changes and commit them to version control.
    3. Verify: Run a read-only check to see if the generated code matches the templates.

    Note: The generator does not create require statements; you must manually add the required namespaces to your template.

    bb apply-import-vars gen-code
    bb apply-import-vars check
  6. Understand rewrite-clj versioning

    main

    The library follows a specific versioning scheme: major.minor.release-test-qualifier.

    • major: Incremented when a non-alpha release API is broken.
    • minor: Incremented when significant new features are added.
    • release: Indicates small changes or bug fixes. Starting from v1.1, this represents the total release count over the life of the project.
    • test-qualifier: Present in non-stable releases (e.g., alpha, beta, rc1).
  7. How auto-resolve affects zipper operations

    main

    When a zipper is created with an :auto-resolve option, the resolution logic is automatically applied during several key operations:

    • sexpr: The current node is converted to its Clojure form using the resolver.
    • find-value and find-next-value: sexpr is applied to each node to retrieve its "value" for comparison.
    • edit: The current node is processed via sexpr.
    • get and assoc: sexpr is applied to the map key being accessed or associated.
  8. Understand S-expression (sexpr) nuances

    main

    Converting rewrite-clj nodes to Clojure forms via z/sexpr or n/sexpr is convenient but has specific behaviors:

    1. Whitespace Loss: Converting to an s-expression strips all original whitespace and comment information.
    2. Non-sexpr-able elements: Certain elements cannot be converted to Clojure forms and will throw an exception if sexpr is called on them. These include:
      • Reader ignore/discard nodes (#_)
      • Comment nodes (;; ...)
      • Whitespace nodes
      • Unbalanced maps (e.g., {:a 1 :b}) or invalid metadata/escaped characters.

    Use sexpr-able? (available in both zip and node APIs) to check if a node can be safely converted before calling sexpr.

    (require '[rewrite-clj.node :as n]
             '[rewrite-clj.parser :as p]
             '[rewrite-clj.zip :as z])
    
    ;; Checking sexpr-ability
    (-> "#_42" z/of-string z/sexpr-able?) ;; => false
    (-> ";; comment" z/of-string z/sexpr-able?) ;; => false
    
    ;; Handling non-sexpr-able nodes safely
    (try
      (-> "#_42" z/of-string z/sexpr)
      (catch ExceptionInfo e (ex-message e)))
  9. Handling location metadata in rewrite-clj

    main

    When coercing Clojure forms to rewrite-clj nodes, the library intentionally omits location metadata (like :line and :column) that Clojure might automatically add (e.g., to quoted lists).

    • No rewrite-clj metadata node is created if the resulting metadata is empty.
    • For compatibility with sci, rewrite-clj also removes :end-line and :end-column metadata.
    • Note that while converting rewrite-clj nodes back to Clojure forms via sexpr, there is currently no way to omit the location metadata.
  10. How namespaced map context is applied

    main

    In rewrite-clj, namespaced map context (e.g., #:prefix) is automatically applied to symbols and keywords within that map. This ensures that when you call sexpr on a key inside a namespaced map, you get the fully qualified Clojure form.

    Key Behaviors:

    • Automatic Application: Context is applied at parse time and whenever a namespaced map node's children are replaced.
    • Zipper Integration: Updates to the map prefix (e.g., replacing the qualifier node) automatically reapply the new context to all children when moving up through the zipper.
    • Manual Reapplication: If you need to manually apply context from the current node downward, use the rewrite-clj.zip/reapply-context function.

    Limitations:

    • Keyword and symbol nodes retain their namespaced map context even if they are moved outside of the map.
    • When working directly with the node API (instead of the zip API), context is only applied at parse time or when children are explicitly replaced.
    (require '[rewrite-clj.zip :as z])
    (require '[rewrite-clj.node :as n])
    
    (def s "#:prefix {:a 1 :b 2 c 3}")
    
    ;; Replacing the prefix reapplies context to children after moving up
    (-> s
        z/of-string
        z/down
        (z/replace (n/map-qualifier-node false "my-new-prefix"))
        z/up
        z/sexpr)
    ;; => #:my-new-prefix{:b 2, c 3, :a 1}
  11. How the Zip API works

    main

    The rewrite-clj.zip namespace is the primary API for traversing and modifying Clojure, ClojureScript, or EDN source code. It uses a customized version of Clojure's clojure.zip.

    A zipper (often named zloc) holds two things:

    1. A tree of rewrite-clj nodes representing the parsed source.
    2. Your current location within that tree.

    Because the zipper is immutable, any movement or modification returns a new zipper instance.

    Note on Navigation: Standard movement functions like right, left, up, and down automatically skip over whitespace and comment nodes. To navigate over every single node (including whitespace and comments), use the * counterparts: right*, left*, up*, and down*.

    (require '[rewrite-clj.zip :as z])
    
    (def data-string "(defn my-function [a] (* a 3))")
    (def zloc (z/of-string data-string))
    
    ;; Navigate and edit
    (-> zloc
        z/down
        z/right
        (z/edit (comp symbol str) "2")
        z/up
        z/sexpr)
    ;; => (defn my-function2 [a] (* a 3))
  12. Understand differences between Clojure and ClojureScript APIs in rewrite-clj v1

    main

    When using rewrite-clj v1, be aware of the following functional and structural differences between the Clojure and ClojureScript implementations:

    • File System Access: The Clojure API includes capabilities for dealing with files directly. The ClojureScript API does not support file system operations.
    • Namespace Availability: The ClojureScript API excludes certain Clojure namespaces that would otherwise cause namespace clashes on the ClojureScript side.
    • Undocumented Features: While many differences are due to the points above, some discrepancies exist because certain internal, undocumented features (functions marked with no-doc) are available in both versions to maintain compatibility with existing usage in rewrite-clj and rewrite-cljs.