xforms

xforms categorizes its transducers into three main groups based on their behavior:

1. Regular transducers: Single-argument transducers like partition, reductions, for, take-last, drop-last, sort, sort-by, wrap, window, and window-by-time.
2. Higher-order transducers: Transducers that accept other transducers as arguments, such as by-key, into-by-key, multiplex, transjuxt, partition (with 2+ args), and time.
3. Aggregators: Transducers that emit exactly one item regardless of input size. These are typically used within higher-order transducers. Examples include reduce, into, without, transjuxt, last, count, avg, sd, min, minimum, max, maximum, and str.

Additionally, net.cgrand.xforms.io provides sh for using processes as reducible collections or transducers.

Xforms supports a mechanism to process key-value pairs without allocating vectors or map entries by implementing the KvRfable protocol (often via the kvrf macro). This allows several transducers and transducing contexts to leverage Clojure's reduce-kv internally.

Key interactions include:

• x/for: When the first binding is a pair and the body-expr is a pair.
• x/reduce: When the reducing function f is a kvrf.
• x/into (transducer): When the destination to is a map.
• x/into (transducing context): When from is a map and to is a map.
• x/by-key (transducer): When kfn and vfn are unspecified or nil.

Using x/into with x/for in a map context prevents unnecessary pair allocations, significantly improving performance.

To use the library in your Clojure or ClojureScript code, require the net.cgrand.xforms namespace:

(require '[net.cgrand.xforms :as x])

If you are using xforms with ClojureScript and the Emacs editor (Figwheel REPL), you may encounter an issue where all REPL results are returned as Strings (e.g., 1 becomes "1").

To fix this, ensure cider.nrepl/cider-middleware is included in your Figwheel's nrepl-middleware configuration:

:figwheel {
  :nrepl-middleware [
    cider.nrepl/cider-middleware
    refactor-nrepl.middleware/wrap-refactor
    cemerick.piggieback/wrap-cljs-repl
  ]
}

The x/window transducer efficiently computes an accumulator over a sliding window of items. It takes the window size, a reducing function, and an optional function to update the accumulator.

;; Sum of last 3 items
(sequence (x/window 3 + -) (range 16))

;; Average of last 4 items
(sequence (x/window 4 rf/avg #(rf/avg %1 %2 -1)) nums)

The x/time higher-order transducer allows you to measure the time spent in a specific transducer (excluding downstream time). The first argument can be a function that receives the elapsed time in milliseconds, allowing for custom logging.

(time ; standard Clojure time
  (count (into [] (comp
                    (x/time "mapinc" (map inc))
                    (x/filterodd (filter odd?))) 
          (range 1e6))))

The x/by-key transducer allows you to group elements by a key function. It is most effective when paired with an aggregator (like x/into or x/reduce) as the last argument to process each partition.

;; Reimplementing group-by using x/by-key and x/reduce
(defn my-group-by [kfn coll]
  (into {} (x/by-key kfn (x/reduce conj)) coll))

;; Reimplementing group-by using x/by-key and x/into (transient version)
(defn my-group-by [kfn coll]
  (into {} (x/by-key kfn (x/into [])) coll))

The x/transjuxt transducer allows you to perform several different transductions (often aggregations) in a single pass over the data. You can pass a vector of transducers or a map of transducers to produce a map of results.

;; Using a vector of transducers
(into {} (x/by-key odd? (x/transjuxt [(x/reduce +) x/avg])) (range 256))

;; Using a map of transducers for named results
(into {} (x/by-key odd? (x/transjuxt {:sum (x/reduce +) :mean x/avg :count x/count})) (range 256))

The x/partition transducer splits a collection into chunks. Like by-key, it accepts a transducer as its last argument to further process each partition. Note that transformed outputs are interleaved.

;; Simple partition with a reduction
(sequence (x/partition 4 (x/reduce +)) (range 16))
;; => (6 22 38 54)

;; Partition with padding
(sequence (x/partition 4 4 (repeat :pad) (x/into [])) (range 9))
;; => ([0 1 2 3] [4 5 6 7] [8 :pad :pad :pad])

The x/for transducer is the transducing version of clojure.core/for. It can be used within a reduce or transduce call for high performance, or used directly which expands to an eduction.

;; Using x/for in a reduction
(reduce + (x/for [i (range 128) j (range i)] (* i j)))

;; Using x/for as an eduction (similar to clojure.core/for)
(x/for [i (range 128) j (range i)] (* i j))