java.util.Map 的现有基于文件的实现

发布于 2024-11-03 07:03:45 字数 258 浏览 8 评论 0原文

我正在开发一个使用自定义 Map的项目。 (其中 Entry 是一对整数)基于 B 树的实现来存储 10 到 1 亿条记录,此类的代码又慢又脏。我需要高效地实现 Map,它使用文件进行存储和少量内存。

我搜索并发现Java版的Berkeley DB有java.util.Collection API(包括Map),但为此目的使用成熟的数据库似乎是多余的(它使用包含许多文件的目录,有几个额外的管理线程) 。有更简单的解决方案吗?

I'm working on a project that uses custom Map<String, Entry> (where Entry is a pair of ints) implementation based on B-tree to store from 10 to 100 millions of records, the code for this class is slow and dirty. I need efficient implementation of the Map, which uses a file for storage and a small amount of memory.

I searched and found that Java Edition Of Berkeley DB has java.util.Collection API (including Map), but it seems superfluous to use a fully fledged database for this purpose (it uses directory with many files, has several additional threads for management). Is there a simpler solution?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

假扮的天使 2024-11-10 07:03:45

我最近遇到了同样的问题,并研究了一切,包括 NoSQL 和缓存。您需要一个基于磁盘/文件/支持的哈希图。

Berkeley DB Java 版是迄今为止最好的。它速度快、可扩展且完整,但如果不分发源代码或从 Oracle 购买商业版本,则无法将其分发给客户。

除了重新发明轮子之外,唯一的选择是 JDBM2。它还具有哈希图和树图。您负责定期刷新到磁盘以防止 OutOfMemoryError,它的速度不如 Berkeley DB,但它是一个非常好的第二选择。

I had this very same problem recently and looked at everything under the sun, including NoSQL and caches. You want a disk/file based/backed hashmap.

Berkeley DB Java Edition is by far the best. It's fast, scalable, and complete, but you can't distribute it to clients without distributing your source code or buying the commercial version from Oracle.

The only other choice, besides reinventing the wheel, is JDBM2. It also has a hashmap and a tree map. You are responsible for regularly flushing to disk to prevent OutOfMemoryError and it isn't near as fast as Berkeley DB but it is a very good 2nd choice.

醉南桥 2024-11-10 07:03:45

看一下 Kyoto Cabininet,一个磁盘支持的 DBM 实现。我使用过以前的版本,Tokyo Cabinet - 它非常容易使用,基本上就像原生< code>Map,而且速度非常快。

Take a look at Kyoto Cabininet, a disk-backed DBM implementation. I've used the previous version, Tokyo Cabinet - it was dead easy to use, basically just like a native Map, and very fast.

烟燃烟灭 2024-11-10 07:03:45

JDBM 是一个轻量级、纯 Java B-Tree 实现。

JDBM is a lightweight, pure Java B-Tree implementation.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文