Haskell 有反向模式自动微分的工作实现吗？
我见过的 Haskell 中最接近相关的实现是 http://hackage.haskell.org/packages/archive/fad/1.0/doc/html/Numeric-FAD.html。
最接近的相关研究似乎是与Scheme相关的另一种功能语言的反向模式，位于http:// /www.bcl.hamilton.ie/~qobi/斯大林格勒/。
我将 Haskell 中的反向模式视为许多任务的圣杯，希望它可以使用 Haskell 的嵌套数据并行性在繁重的数值优化中获得不错的加速。
The closest-related implementation in Haskell I have seen is the forward mode at http://hackage.haskell.org/packages/archive/fad/1.0/doc/html/Numeric-FAD.html.
The closest related related research appears to be reverse mode for another functional language related to Scheme at http://www.bcl.hamilton.ie/~qobi/stalingrad/.
I see reverse mode in Haskell as kind of a holy grail for a lot of tasks, with the hopes that it could use Haskell's nested data parallelism to gain a nice speedup in heavy numerical optimization.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
为了回答这个问题，我上传了一个名为 ad 的包到 Hackage 来处理反向模式Haskell 中的自动微分。
在内部，它利用 Andy Gill 的 Kansas Lava 的技巧来观察其记录的磁带中的共享，以用于反向传播，并使用类型级别品牌来避免混淆敏感性。
我试图让 API 相对接近 Barak Pearlmutter 和 Jeffrey Mark Siskind 的时尚包，但为了通用性，我忍不住到处做了一些小调整。
我仍然需要仔细研究并完成剩余的未实现的时尚组合器，找出构建反向模式 AD 塔的好方法，验证我没有搞砸我对基本微积分的记忆，并提供一个很好的 API 来使用这种方法可以在其他正向模式 AD 程序中获取本地反向模式检查点，但我对迄今为止的进展感到非常满意。
In response to this question, I've uploaded a package named ad to Hackage for handling reverse-mode automatic differentiation in Haskell.
Internally, it leverages a trick from Andy Gill's Kansas Lava to observe sharing in the tape it records for back propagation purposes, and uses type level branding to avoid confusing sensitivities.
I've tried to keep the API relatively close to that of Barak Pearlmutter and Jeffrey Mark Siskind's fad package, but I couldn't resist making a couple of minor tweaks here and there for generality.
I still need to go through and finish up the remaining unimplemented fad combinators, figure out a nice way to build a reverse-mode AD tower, validate that I didn't screw up my recollection of basic calculus, and provide a nice API for using this approach to get local reverse mode checkpoints in an otherwise forward mode AD program, but I am quite happy with how things have progressed thus far.
我们有一堆正向模式 AD 实现（我什至在我的幺半群库中也有一个！），但对于所有 Haskell 来说反向模式 AD 似乎很棘手。
遗憾的是，虽然 Pearlmutter 和 Siskind 给出了 lambda 演算的翻译，但它并没有映射到您可以对任意 Haskell lambda 执行的操作，您没有获得正确的内省属性，并且考虑到类型形状在翻译中发生变化的方式你不会得到适合被打包到 monad、箭头或其他控制结构中的东西。
我通过与 Pearlmutter 的一系列电子邮件交流进行了尝试，但最终我能够获得的最好的结果是 Haskell 中小型 EDSL 的反向模式 AD 解决方案，而不是 Haskell 本身的解决方案。
We have a bunch of forward mode AD implementations (I even have one in my monoids library!), but reverse mode AD for all of Haskell seems to be intractable.
Sadly while Pearlmutter and Siskind give a translation for a lambda calculus, it doesn't map into something you can do for arbitrary Haskell lambdas, you don't get the right introspection properties and given the way the shape of the types change in the translation you don't get something that is amenable to being packed into a monad, arrow, or other control structure.
I had a go at it via a series of email exchanges with Pearlmutter, but ultimately the best I was able to obtain was a reverse mode AD solution for a small EDSL in Haskell, and not a solution for Haskell itself.
据我所知。我确实知道一些 Haskell 各位 是 对自动微分感兴趣，但一些快速挖掘发现除了提到反向模式的简短旁白之外什么也没有；我希望您已经找到了与我相同的材料。
我还注意到，您发现的
fad包和 Stalingrad 项目实际上是同一个 两个 人，而且至少教授。 Pearlmutter 已发布到 haskell-cafe 邮件列表。您可能需要考虑直接联系他了解他的工作——他可能正在做一些事情，或者在尝试实施反向模式 AD 时遇到严重障碍。抱歉，我无法找到更有用的东西；如果其他人想进一步挖掘，至少上面的链接是一个起点。
Not that I'm aware of. I do know that some Haskell folks are interested in automatic differentiation, but some quick digging found little more than short asides mentioning the reverse mode; I expect you already found the same material I did.
I also note that the
fadpackage and Stalingrad project you found are in fact the work of the same two people, and that at least Prof. Pearlmutter has posted to the haskell-cafe mailing list. You may want to consider contacting him directly about his work--it's possible he has something in progress, or hit serious obstacles while attempting to implement reverse-mode AD.Sorry I couldn't turn up anything more useful; if someone else wants to dig further, at least the links above are a starting point.
我认为前进是 Haskell 的出路。正如爱德华指出的那样，您不应该能够对任意函数执行反向模式。但您回答说您应该能够在某些受限功能上做到这一点。并且所述约束可以很容易地导致前进模式。例如。如果你有一个函数：
那么你可以用可微类型实例化
a，从而在正向模式下区分foo。请参阅 Hackage 上的 vector-space 库，了解非常优雅的前向模式自动微分。一开始可能并不完全清楚如何使用它。请阅读 Conal Elliott 撰写的论文《美丽的差异化》。
I think forward is the way to go in Haskell. You shouldn't be able to do reverse mode on arbitrary functions, as Edward pointed out. But you responded that you should be able to do it on certain constrained functions. And said constraints can lead readily to forward mode. Eg. if you have a function:
Then you can instantiate
awith a differentiable type, and thus differentiatefooin forward mode.See the vector-space library on Hackage for very elegant forward mode automatic differentiation. It might not be entirely clear how to use it at first. Read the paper about it, Beautiful Differentiation by Conal Elliott.