基于磁盘的HashMap

发布于 2024-08-28 19:25:48 字数 1809 浏览 6 评论 0原文

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(8

冰雪梦之恋 2024-09-04 19:25:48

MapDB

MapDB 提供由磁盘存储或堆外内存支持的并发 TreeMap 和 HashMap。它是一个快速、可扩展且易于使用的嵌入式 Java 数据库引擎。它具有事务、节省空间的序列化、实例缓存和透明压缩/加密等功能。它还具有只有本机嵌入式数据库引擎才能媲美的卓越性能。

http://www.mapdb.org/

jdbm2

嵌入式键值 Java 数据库。

https://code.google.com/p/jdbm2/

MapDB

MapDB provides concurrent TreeMap and HashMap backed by disk storage or off-heap-memory. It is a fast, scalable and easy to use embedded Java database engine. It is packed with features such as transactions, space efficient serialization, instance cache and transparent compression/encryption. It also has outstanding performance rivaled only by native embedded db engines.

http://www.mapdb.org/

jdbm2

Embedded Key Value Java database.

https://code.google.com/p/jdbm2/

世界和平 2024-09-04 19:25:48

2018 中,最轻的持久键值存储是 H2 数据库及其 MVStore

MVStore 是一个持久的、日志结构的键值存储。这是
计划成为 H2 的下一个存储子系统,但它也可以
直接在应用程序中使用,无需使用 JDBC 或 SQL。

  • MVStore 代表“多版本商店”。

  • 每个商店都包含许多可以使用 java.util.Map 接口访问的地图。

  • 支持基于文件的持久性和内存中操作。

  • 它的目标是速度快、使用简单且体积小。

  • 支持并发读写操作。

  • 支持事务(包括并发事务和两阶段提交)。

  • 该工具非常模块化。它支持可插拔数据类型和序列化、可插拔存储(到文件、到堆外内存)、可插拔映射实现(B 树、R 树、当前的并发 B 树)、BLOB 存储和文件系统抽象支持加密文件和zip文件。

h2-mvstore 没有依赖项,并且版本 1.4.200 是 0.3 Mb

我还查看了:

  • MapDB13 meg 依赖项)
  • chronicle-map5.5 meg 依赖项 - 快速可选分发)
  • lmdbjava2 meg java 依赖项 + lmdb C 库) - 最快的实现,但不是开箱即用的线程安全

In 2018 the lightest persistent key value store is the H2 Database with it's MVStore:

The MVStore is a persistent, log structured key-value store. It is
planned to be the next storage subsystem of H2, but it can also be
used directly within an application, without using JDBC or SQL.

  • MVStore stands for "multi-version store".

  • Each store contains a number of maps that can be accessed using the java.util.Map interface.

  • Both file-based persistence and in-memory operation are supported.

  • It is intended to be fast, simple to use, and small.

  • Concurrent read and write operations are supported.

  • Transactions are supported (including concurrent transactions and 2-phase commit).

  • The tool is very modular. It supports pluggable data types and serialization, pluggable storage (to a file, to off-heap memory), pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.

h2-mvstore has no dependencies and version 1.4.200 is a single jar of 0.3 Mb

I also looked at:

  • MapDB (13 meg dependencies)
  • chronicle-map (5.5 meg dependencies - fast optionally distributed)
  • lmdbjava (2 meg java dependencies + lmdb C library) - fastest implementation but not thread safe out of the box.
一个人练习一个人 2024-09-04 19:25:48

属性文件Berkeley DB 可能就是您要找的。 java.util.Properties 本身实现了java.util.Map,并提供了加载存储到文件的方法。 Berkeley DB 通常被推荐作为轻量级键值对数据存储。

Either properties files or Berkeley DB might be what you're looking for. The java.util.Properties itself implements java.util.Map and provides methods to load from and store to a file. The Berkeley DB is often been recommended as a lightweight key-value pair datastore.

倾城泪 2024-09-04 19:25:48

听起来你需要一些接近轻量级数据库的东西。您是否看过/考虑过 Java DB?单个索引表基本上是基于磁盘的线程安全哈希映射。

Sounds like you need something close to a lightweight db. Have you looked at/considered Java DB? A light db with a single, indexed table would basically be a disk-based, thread-safe hash map.

冰雪梦之恋 2024-09-04 19:25:48

JDBM2 正是您所要求的。它提供了一个由磁盘存储备份的 HashMap(以及其他映射)。它快速、线程安全并且 API 非常简单。

JDBM2 is exactly what you are asking. It provides a HashMap backed up by disk storage (among other maps). Its fast, thread-safe and the API is really simple.

独木成林 2024-09-04 19:25:48

Chronicle Map 实现 ConcurrentMap 并保存数据通过将内存映射到文件到磁盘。

Chronicle Map 在概念上与 MapDB 非常相似(提供类似的构建器 API 和 Map 接口),但 Chronicle Map 是 比 MapDB 快 倍,并且具有更好的并发性(Chronicle Map 使用高度条带化的多级自旋锁)。

Chronicle Map implements ConcurrentMap and persists data to disk via mapping it's memory to a file.

Chronicle Map is conceptually very similar to MapDB (provides similar builder API and Map interface), but Chronicle Map is times faster than MapDB and has much better concurrency (Chronicle Map uses highly striped multi-level spin locks).

夏九 2024-09-04 19:25:48

Project Voldemort 也是一个非常快速/可扩展/复制的“哈希映射”。它在 LinkedIn 上使用,性能也相当不错:

来自他们网站的引用:

这是我们从
与单个多线程客户端通信
“热”数据所在的单个服务器
人为设置在内存中
我们的性能实验室的重负载:

读取:19,384 请求/秒
写入:16,559 请求/秒

Project Voldemort is also a really fast/scalable/replication "Hashmap". It is used at LinkedIn an performance is also pretty good:

A quote from their site:

Here is the throughput we see from a
single multithreaded client talking to
a single server where the "hot" data
set is in memory under artificially
heavy load in our performance lab:

Reads: 19,384 req/sec
Writes: 16,559 req/sec

深居我梦 2024-09-04 19:25:48

现在是 2016 年。如果有人想解决这个问题,我发现 Xodus 中的低级环境 API JetBrains 的 使用其 computeInTransaction 存储 lambda 来实现相同的目的。

当然,它不像纯 Map 实例那么灵活,但它适用于我的用例。

最近的另一个选择是使用 H2 的 MVStore 存储引擎同样的事情,但我认为它更适合数据库本身。

干杯!

So the year is now 2016. And if anyone's looking to tackle this problem, I found out that the low level environments API in Xodus from JetBrains works for this same purpose, using their computeInTransaction store lambdas.

Granted, it's not as slick as having a pure Map instance, but it worked for my use case.

Another recent option is to use H2's MVStore storage engine which does the same thing, but I think it's more tailored towards the database itself.

Cheers!

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文