在 clojure 中,通过内核对向量进行卷积的有效方法是什么?

发布于 2024-12-17 15:26:59 字数 722 浏览 0 评论 0原文

我想出了这个:

(def kernel [0 1 1 2 3 3 0 0 0 0 0 0])
(def data [1 5 7 4 8 3 9 5 6 3 2 1 1 7 4 9 3 2 1 8 6 4])

(defn capped+ [a b c] (let [s (+ a b)] (if (> s c) c s)))

(defn *+ [a b]
  (if (> (count a) (count b))  
    (reduce + (map-indexed (fn _ [i x] (* (a i) (b i))) b))
    (reduce + (map-indexed (fn _ [i x] (* (a i) (b i))) a))))

(defn slide*i [d k] 
 (let [ki (into [] (reverse k)) kl (count k) dl (count d)]
   (map-indexed 
      (fn [idx item] 
        (/ (*+ ki (subvec d idx (capped+ idx kl dl))) 
           (reduce + ki))) 
      d)))

(def s (slide*i data kernel))

这不是最优雅的代码,但它工作得很好。 我实际上想用它来平滑一些非常尖的!数据。

欢迎任何使之更美观、更高效或更准确的建议(我个人并不关心尾巴不准确,因为就我而言,我从不使用它)。

I came up with this:

(def kernel [0 1 1 2 3 3 0 0 0 0 0 0])
(def data [1 5 7 4 8 3 9 5 6 3 2 1 1 7 4 9 3 2 1 8 6 4])

(defn capped+ [a b c] (let [s (+ a b)] (if (> s c) c s)))

(defn *+ [a b]
  (if (> (count a) (count b))  
    (reduce + (map-indexed (fn _ [i x] (* (a i) (b i))) b))
    (reduce + (map-indexed (fn _ [i x] (* (a i) (b i))) a))))

(defn slide*i [d k] 
 (let [ki (into [] (reverse k)) kl (count k) dl (count d)]
   (map-indexed 
      (fn [idx item] 
        (/ (*+ ki (subvec d idx (capped+ idx kl dl))) 
           (reduce + ki))) 
      d)))

(def s (slide*i data kernel))

It's not the most elegant code, but it works fine.
I actually want to use it to smooth some very spiky! data.

Any suggestions to make this more beautiful or more efficient or more accurate (personally I don't care about the tail being inaccurate because in my case I never use it) are welcomed.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

孤云独去闲 2024-12-24 15:26:59

您当然可以显着提高此操作的性能。好消息是,您不需要为此投入 Java:如果正确优化,Clojure 会非常快,并且在大多数情况下可以产生与纯 Java 相同的速度。

为了在 Clojure 中实现数字代码的最大性能,您将需要使用:

  • 数组,因为您需要具有非常快的写入和查找速度的可变存储。 Clojure 序列和向量很漂亮,但它们会带来一些开销,对于真正的性能关键型代码
  • double 原语,您可能希望避免这些开销,因为它们提供更快的数学运算。
  • aset / aget / areduce - 这些是专为数组设计的极其快速的操作,基本上为您提供与纯 Java 等效项相同的字节码。
  • 命令式风格 - 虽然它在 Clojure 中不惯用,但它获得最快的结果(主要是因为您可以避免内存分配、装箱和函数调用的开销)。一个例子是使用 dotimes 来实现快速命令式循环。
  • (set! *warn-on-reflection* true) - 并消除代码产生的任何警告,因为反射是一个很大的性能杀手。

以下内容应该是正确的,并且可能会为您提供与 Java 大致相同的性能:

(def kernel (double-array [0 1 1 2 3 3 0 0 0 0 0 0]))
(def data (double-array [1 5 7 4 8 3 9 5 6 3 2 1 1 7 4 9 3 2 1 8 6 4]))

(defn convolve [^doubles kernel-array ^doubles data-array]
  (let [ks (count kernel-array)
        ds (count data-array)
        output (double-array (+ ks ds))
        factor (/ 1.0 (areduce kernel-array i ret 0.0 (+ ret (aget kernel-array i))))]    
    (dotimes [i (int ds)]
      (dotimes [j (int ks)]
        (let [offset (int (+ i j))]
          (aset output offset (+ (aget output offset) (* factor (* (aget data-array i) (aget kernel-array j))))))))
    output))

(seq (convolve kernel data))
=> (0.0 0.1 0.6 1.4 2.4 4.4 5.5 6.1000000000000005 5.600000000000001 6.200000000000001 5.499999999999999 5.9 4.199999999999999 3.3000000000000003 2.5 2.2 3.3 4.4 5.6000000000000005 4.8 4.8999999999999995 3.1 3.5 4.300000000000001 5.0 3.0 1.2000000000000002 0.0 0.0 0.0 0.0 0.0 0.0 0.0)

我没有修剪输出数组或进行任何限制,因此您可能需要稍微修改此解决方案才能准确获得您想要的输出,但希望你能明白......

一些非常粗略的基准测试:

(time (dotimes [i 1000] (seq (convolve kernel data))))
=> "Elapsed time: 8.174109 msecs"

即每个内核/数据对组合大约 30 纳秒 - 我预计这几乎达到了缓存内存访问的界限。

You can certainly improve the performance of this operation significantly. The good news is that you don't need to drop into Java for this: Clojure is extremely fast if you optimise it correctly and in most instances can produce the same speed as pure Java.

For maximum performance of numerical code in Clojure you will want to use:

  • arrays, because you want mutable storage with very fast writes and lookup. Clojure sequences and vectors are beautiful, but they come with overheads that you probably want to avoid for truly performance-critical code
  • double primitives, because they offer much faster maths.
  • aset / aget / areduce - these are extremely fast operations designed for arrays and basically give you the same bytecode as pure Java equivalents.
  • imperative style - although it's unidiomatic in Clojure, it gets the fastest results (mainly because you can avoid overheads from memory allocations, boxing and function calls). An example would be using dotimes for a fast imperative loop.
  • (set! *warn-on-reflection* true) - and eliminate any warnings your code produces, because reflection is a big performance killer.

The following should be along the right lines and will probably get you roughly equivalent performance to Java:

(def kernel (double-array [0 1 1 2 3 3 0 0 0 0 0 0]))
(def data (double-array [1 5 7 4 8 3 9 5 6 3 2 1 1 7 4 9 3 2 1 8 6 4]))

(defn convolve [^doubles kernel-array ^doubles data-array]
  (let [ks (count kernel-array)
        ds (count data-array)
        output (double-array (+ ks ds))
        factor (/ 1.0 (areduce kernel-array i ret 0.0 (+ ret (aget kernel-array i))))]    
    (dotimes [i (int ds)]
      (dotimes [j (int ks)]
        (let [offset (int (+ i j))]
          (aset output offset (+ (aget output offset) (* factor (* (aget data-array i) (aget kernel-array j))))))))
    output))

(seq (convolve kernel data))
=> (0.0 0.1 0.6 1.4 2.4 4.4 5.5 6.1000000000000005 5.600000000000001 6.200000000000001 5.499999999999999 5.9 4.199999999999999 3.3000000000000003 2.5 2.2 3.3 4.4 5.6000000000000005 4.8 4.8999999999999995 3.1 3.5 4.300000000000001 5.0 3.0 1.2000000000000002 0.0 0.0 0.0 0.0 0.0 0.0 0.0)

I've not trimmed the output array or done any bounding so you'll probably need to hack this solution a bit to get exactly the output you want, but hopefully you get the idea.....

Some very rough benchmarking:

(time (dotimes [i 1000] (seq (convolve kernel data))))
=> "Elapsed time: 8.174109 msecs"

i.e. that's about 30ns per kernel / data pair combination - I expect that's pretty much hitting the bounds of cached memory access.

長街聽風 2024-12-24 15:26:59
; unused, but left for demonstration
(defn capped+ [a b c]
  (min (+ a b) c))

(defn *+ [a b]
  (reduce + (map * a b)))

(defn slide*i [d k]
  (let [ki (reverse k)
        kl (count k)
        ks (reduce + k)]
    (map #(/ (*+ %1 ki) ks) (partition kl 1 d))))

使用partition,结果是:

(59/10 21/5 33/10 5/2 11/5 33/10 22/5 28/5 24/5 49/10 31/10)

但是使用partition-all,您将得到您的解决方案的结果:

(59/10 21/5 33/10 5/2 11/5 33/10 22/5 28/5 24/5 49/10 31/10 7/2 43/10 5 3 6/5 0 0 0 0 0 0)
; unused, but left for demonstration
(defn capped+ [a b c]
  (min (+ a b) c))

(defn *+ [a b]
  (reduce + (map * a b)))

(defn slide*i [d k]
  (let [ki (reverse k)
        kl (count k)
        ks (reduce + k)]
    (map #(/ (*+ %1 ki) ks) (partition kl 1 d))))

With partition, the results are:

(59/10 21/5 33/10 5/2 11/5 33/10 22/5 28/5 24/5 49/10 31/10)

But with partition-all, you'll get exactly what your solution resulted in:

(59/10 21/5 33/10 5/2 11/5 33/10 22/5 28/5 24/5 49/10 31/10 7/2 43/10 5 3 6/5 0 0 0 0 0 0)
朕就是辣么酷 2024-12-24 15:26:59

执行此操作的有效方法是创建执行卷积的 java 类并从 clojure 调用它,如果可能的话向其传递一个 java 数组。如果考虑效率的话,Clojure 实现也应该在 java 数组上运行。

The efficient way of doing this is to create java class that does convolution and call it from clojure, passing it a java array if possible. Clojure implementation should operate on java arrays as well if efficiency is a concern.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文