Stateful transducers, thread safety and performance

Question

A quote from https://clojure.org/reference/transducers#_creating_transducible_processes:

A transducing process must encapsulate references to the function
returned by invoking a transducer - these may be stateful and unsafe
for use across threads.

1. Does it mean that it is OK to create unsafe transducers for better
   performance?
2. Can core's stateful transducers be written unsafe
   for having performance close to loop/recur?

Benchmarks for unsafe implementation of clojure.core/drop:

(definterface IMutable
  (get [])
  (set [new-val]))

(deftype UnsynchronizedMutable [^:unsynchronized-mutable n]
  IMutable
  (get [_] n)
  (set [_, nv] (set! n nv)))

(defn drop!
  ([n]
   (fn [rf]
     (let [nv (UnsynchronizedMutable. n)]
       (fn
         ([] (rf))
         ([result] (rf result))
         ([result input]
          (let [^long n (.get nv)]
            (.set nv (dec n))
            (if (pos? n)
              result
              (rf result input)))))))))

(comment
  (criterium.core/quick-bench
    (transduce (comp
                 (map inc)
                 (drop 1))
      + (range 1000)))
  #_"Execution time mean : 39,455570 µs"
  
  (criterium.core/quick-bench
    (transduce (comp
                 (map inc)
                 (drop! 1))
      + (range 1000)))
  #_"Execution time mean : 24,979885 µs")

Answer 1

The answer can be found in this discussion https://groups.google.com/d/msg/clojure/VQj0E9TJWYY/nXVf_H6sAgAJ

So yes, we can use unsafe transducers when we are sure about single-threaded reducing context. But not in general case.

Answer 2

If you're willing to fight this battle, you might be interested by this thread https://clojure.atlassian.net/browse/CLJ-2146

Comments

It's a shame that your IMO solid post was not addressed.

---

i second that.

---

I am interested in the details of the JMM, too, but they are fairly subtle, and at least in this case, the Clojure dev team, if they are erring, seem to be erring on the side of maybe sometimes extra safety at the cost of some performance, which seems to be a far preferable choice than erring in the opposite direction.