FORTRAN 或 C（或托管代码）中的数据挖掘模型？
我们计划开发一个 Windows 数据挖掘包。程序核心/计算引擎将使用 F# 开发，GUI 内容/数据库绑定等则使用 C# 和 F# 完成。
但是，我们尚未决定模型的实现。由于我们需要高性能，因此我们可能不能在这里使用托管代码（这里有什么异议吗？）。问题是，用 FORTRAN 开发模型是否合理，还是应该坚持使用 C（或者 C++）。我们正在考虑在某个时候使用 OpenCL 来寻找合适的模型 - 不得不从托管代码转移感觉很有趣 -> FORTRAN-> C->针对这些情况的 OpenCL 调用。
有什么建议吗？
We are planning to develop a datamining package for windows. The program core / calculation engine will be developed in F# with GUI stuff / DB bindings etc done in C# and F#.
However, we have not yet decided on the model implementations. Since we need high performance, we probably can't use managed code here (any objections here?). The question is, is it reasonable to develop the models in FORTRAN or should we stick to C (or maybe C++). We are looking into using OpenCL at some point for suitable models - it feels funny having to go from managed code -> FORTRAN -> C -> OpenCL invocation for these situations.
Any recommendations?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
F# 编译为 CLR，后者具有即时编译器。它是 ML 的一种方言，它是强类型的，允许与该类型的架构一起进行所有良好的优化；这意味着您可能会从 F# 获得合理的性能。为了进行比较，您还可以尝试将代码移植到 OCaml（IIRC 这会编译为本机代码），看看这是否会产生重大差异。
如果它真的太慢，那么看看扩展硬件能让你走多远。凭借现代 PC 或服务器提供的性能，您似乎不太可能需要去任何异国情调的地方，除非您正在使用真正的 brobdinagian 数据集。数据集较小的用户在普通 PC 上可能没问题。
工作站的容量可能比标准台式电脑高一个数量级。高端工作站，例如 HP Z800 或 XW9400（其他几个制造商也提供类似套件）可以占用两个4 或 6 核 CPU 芯片、数十 GB RAM（某些情况下高达 192 GB），并具有多种高速 I/O 选项，如 SAS 磁盘，外部磁盘阵列 或 SSD。这种类型的硬件很昂贵，但可能比大量程序员的时间便宜。您现有的桌面支持基础设施应该能够支持此类套件。最可能的问题是在 64 位操作系统上运行 32 位软件的兼容性问题。在这种情况下，您有多种选择（例如 VM 或 KVM 交换机）来解决兼容性问题。
下一步是 4 或 8 套接字服务器。相当普通的 wintel 服务器最多有 8 个插槽（32-48 个内核），可能还有 512GB RAM - 无需脱离 Wintel 平台。这为您在选择的平台中提供了相当广泛的选择，然后您就必须去任何异国情调1。
最后，如果无法使其在 F# 中快速运行，请验证 F# 原型并使用 F# 原型作为控件构建 C 实现。如果这仍然不够快，那么你就会遇到问题。
如果您的应用程序可以以适合该平台的方式构建，那么您可以考虑一个更奇特的平台。根据您的应用程序的工作方式，您可以将其托管在集群、云提供商上或在 GPU、 Cell 处理器 或 FPGA。 但是，这样做会带来（相当大的）额外成本和可能导致支持问题的外来依赖性。您可能还需要聘请一位知道如何对平台进行编程的第三方顾问。
毕竟，最好的建议是：吸吮它并看看。如果您熟悉 F#，您应该能够相当快速地构建应用程序原型。看看它的运行速度有多快，不要太担心性能，直到你有明确的迹象表明它确实会成为一个问题。请记住，Knuth 说过，过早优化大约 97% 的情况是万恶之源。如果您认为性能确实会造成麻烦，请密切关注天气问题并重新评估您的策略。
编辑：如果您想制作打包应用程序，那么您可能会比其他应用程序对性能更加敏感。在这种情况下，性能可能比定制系统更早成为问题。然而，这并不影响基本的“拭目以待”原则。
但是，在这种类型的架构上，与数据源的互联网连接成为瓶颈。如果您有大量数据集，那么将它们上传到服务提供商就会成为问题。在本地处理大型数据集可能比通过互联网连接上传更快。
F# compiles to the CLR, which has a just-in-time compiler. It's a dialect of ML, which is strongly typed, allowing all of the nice optimisations that go with that type of architecture; this means you will probably get reasonable performance from F#. For comparison, you could also try porting your code to OCaml (IIRC this compiles to native code) and see if that makes a material difference.
If it really is too slow then see how far that scaling hardware will get you. With the performance available through a modern PC or server it seems unlikely that you would need to go to anything exotic unless you are working with truly brobdinagian data sets. Users with smaller data sets may well be OK on an ordinary PC.
Workstations give you perhaps an order of magnitude more capacity than a standard dekstop PC. A high-end workstation like a HP Z800 or XW9400 (similar kit is available from several other manufacturers) can take two 4 or 6 core CPU chips, tens of gigabytes of RAM (up to 192GB in some cases) and has various options for high-speed I/O like SAS disks, external disk arrays or SSDs. This type of hardware is expensive but may be cheaper than a large body of programmer time. Your existing desktop support infrastructure shouldn be able to this sort of kit. The most likely problem is compatibility issues running 32 bit software on a 64-bit O/S. In this case you have various options like VMs or KVM switches to work around the compatibility issues.
The next step up is a 4 or 8 socket server. Fairly ordinary wintel servers go up to 8 sockets (32-48 cores) and perhaps 512GB of RAM - without having to move off the Wintel platform. This gives you fairly wide range of options within your platform of choice before you have to go to anything exotic1.
Finally, if you can't make it run quickly in F#, validate the F# prototype and build a C implementation using the F# prototype as a control. If that's still not fast enough you've got problems.
If your application can be structured in a way that suits the platform then you could look at a more exotic platform. Depending on what will work with your application, you might be able to host it on a cluster, cloud provider or build the core engine on a GPU, Cell processor or FPGA. However, in doing this you're getting into (quite substantial) additional costs and exotic dependencies that might cause support issues. You will probably also have to bring a third-party consultant who knows how to program the platform.
After all that, the best advice is: suck it and see. If you're comfortable with F# you should be able to prototype your application fairly quickly. See how fast it runs and don't worry too much about performance until you have some clear indication that it really will be an issue. Remember, Knuth said that premature optimisation is the root of all evil about 97% of the time. Keep a weather eye out for issues and re-evaluate your strategy if you think performance really will cause trouble.
Edit: If you want to make a packaged application then you will probably be more performance-sensitive than otherwise. In this case performance will probably become an issue sooner than it would with a bespoke system. However, this doesn't affect the basic 'suck it and see' principle.
However, on this type of architecture the internet connection to the data source becomes a bottleneck. If you have large data sets then uploading these to the service provider becomes a problem. It may be quicker to process a large dataset locally than to upload it through an internet connection.
我建议不要为优化而烦恼。首先尝试获得一个工作原型，然后找出计算时间花费在哪里。如果需要的话，您可以将最大的瓶颈转移到 C 或 Fortran 中，然后看看它会产生多大的差异。
正如他们所说，通常 90% 的计算花费在 10% 的代码中。
I would advise not to bother with optimizations yet. First try to get a working prototype, then find out where computation time is spent. You can probably move the biggest bottlenecks out into C or Fortran when and if needed -- then see how much difference it makes.
As they say, often 90% of the computation is spent in 10% of the code.