将映射的 ByteString 转换为其他类型？
我意识到这可能是一个相当异端的问题，但我想知道是否可以通过 System.IO.Posix.MMap 映射数据文件，然后将生成的 ByteString 转换为严格的数组其他类型？例如。如果我知道该文件包含双精度数，我是否可以以某种方式将此映射数据放入 UArr Double 中，以便我可以对其执行 sumU 等操作，并让虚拟内存系统为我处理 IO？这本质上就是我在 C++ 代码中处理多 GB 数据集的方式。也欢迎其他更惯用的方法来做到这一点，谢谢！
对于我也可以对数据进行多核处理的方式的最高加分:-)并不是我要求或任何东西。
I realize this may be a rather heretical question, but I wonder whether I can mmap a file of data, via System.IO.Posix.MMap, and then cast the resulting ByteString into a strict array of some other type? Eg. if I know that the file contains doubles, can I somehow get this mmapped data into an UArr Double so I can do sumU etc on it, and have the virtual memory system take care of IO for me? This is essentially how I deal with multi-GB data sets in my C++ code. Alternative more idiomatic ways to do this also appreciated, thanks!
Supreme extra points for ways I can also do multicore processing on the data :-) Not that I'm demanding or anything.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
我认为这样做不安全。 UArr 是 Haskell 堆分配的未固定内存，GC 将移动它。 ByteStrings（和映射的）是固定内存的ForeignPtr。它们是运行时系统中的不同对象。
如果您要将基础类型从ForeignPtr 更改为Haskell 值“a”，则需要进行复制以确保安全。
I don't think it is safe to do this. UArr are Haskell heap allocated unpinned memory, the GC will move it. ByteStrings (and mmapped ones) are ForeignPtrs to pinned memory. They're different objects in the runtime system.
You will need to copy for this to be safe, if you're changing the underlying type from ForeignPtr to a Haskell value 'a'.
恐怕我不知道如何将
ByteString转换为UArr T，但我想通过建议您采取一些“额外的积分”来主张查看数据并行Haskell；从您所描述的问题来看，它可能就在您的街道上。I'm afraid I don't know how to cast a
ByteStringto aUArr T, but I'd like to claim some "extra points" by suggesting you take a look at Data Parallel Haskell; from the problem you've described it could be right up your street.您可能需要Foreign.Marshal在这里，尤其是Foreign.Marshal.Array。它的设计正是为了做到这一点。
You probably want Foreign.Marshal here, and especially Foreign.Marshal.Array. It was designed to do exactly this.