当前位置：文江博客话题详情

Java嵌入式DB，不需要JDBC

发布于 2024-09-19 17:51:11 字数 1395 浏览 6 评论 0原文

我有以下要求：

在磁盘上保留用户列表以供将来加载。每个用户可能有 3 或 4 个属性。数十到数百个用户是典型的，必须支持数千个用户（不需要支持超过 10,000 个用户）。
在磁盘上保留记录列表以供将来加载。每条记录可能有十几个属性。通常有 0 到 10 条记录，但必须支持数千条记录数（以处理由于网络故障等原因而在一段时间内未处理记录的情况）。

用户通常作为一批写入，然后定期搜索以查找具有给定属性的用户（例如，用于通过提供的密码对用户进行身份验证）。

记录是间歇性写入和间歇性读取的（后者由周期性任务在删除记录之前处理和传输它们）。

我面临以下限制：

这是一个嵌入式设备，仅支持 Java 1.1.8/1.2.x 的子集。包含的软件包如下：
- java.lang
- java.io
- java.util
- java.net
- java.lang.reflect
- java.util.zip
- java.math
- java.text
- java.security
这些设备拥有相当适中的资源（例如~20 MB RAM，具体取决于设备），如果可行的话，最好将它们存储在磁盘上而不是内存中。
我们对设备的访问受到限制，我们的应用程序被放置在自己的沙箱中，使得完整的数据库安装不可行。我们的沙箱内确实有磁盘访问权限。
只有单个应用程序需要访问此信息，并且访问可以同步，这意味着线程安全/并发访问不一定是必需的。

我们有一个与我正在开发的应用程序类似的应用程序，但针对不同的设备，该设备使用用户名的专有文本格式（例如散列分隔）和记录的 ObjectOutputStream。

我认为当前实现的缺点是：

整个文件作为一个整体进行读取或写入，这意味着必须经常读取文件，或者必须保留数据的内存副本，并且仅在需要时才将其写回磁盘。变化。当像当前应用程序那样选择后一种选择时，这意味着内存使用量可以根据数据的大小无限增长。
当前的两种格式都是专有的，前者容易产生错误数据（当用户名中包含哈希值时会发生什么？），而后者不可读（或可通过常用工具查询）。

这似乎是一个简单的基于文件的嵌入式数据库是理想的情况，例如 derby 或 < a href="http://www.zentus.com/sqlitejdbc/" rel="nofollow noreferrer">sqlite。然而，从我迄今为止的研究看来，大多数选项都涉及 JDBC 驱动程序，而我没有可用的驱动程序（该设备上未实现 java.sql.*）。

有人知道现有的项目很适合吗？

原文

I have the following requirements:

Keeping a list of users on disk for future loading. Each user has maybe 3 or 4 properties. Dozens to hundreds of users is typical, thousands of users must be supported (support for more than say 10,000 is not necessary).
Keeping a list of records on disk for future loading. Each record has probably a dozen or so properties. There will generally be between 0 and 10 records, but number of records into the low thousands must be supported (to handle the case where records are not being processed for a period of time because of, e.g., the network being down).

The users are generally written as a batch and then periodically searched to find one with a given property (e.g. for authenticating a user via a supplied password).

The records are written intermittently and read intermittently (the latter by a periodic task which processes and transmits them before deleting the record).

I face the following limitations:

This is an embedded device which supports only a subset of Java 1.1.8/1.2.x. The included packages are as follows:
- java.lang
- java.io
- java.util
- java.net
- java.lang.reflect
- java.util.zip
- java.math
- java.text
- java.security
The devices have fairly modest resources (e.g. ~20 MB RAM depending on the device) making it desirable to store these on disk rather than in memory if feasible.
We have limited access to the device with our application being placed in its own sandbox, making a full-blown database install infeasible. We do have disk access within our sandbox.
Only a single application needs access to this information and access can be synchronized, meaning thread-safety/concurrent access is not necessarily a requirement.

We have an application similar to the one I'm developing but for a different device which uses a proprietary text format for the usernames (e.g. hash delimited) and an ObjectOutputStream for the records.

The downsides I see to the current implementation:

The entire file is read or written at as a whole, meaning that either the files must be read frequently or an in-memory copy of the data must be kept and only written back to disk when it changes. When going with the latter choice as the current application does it means that memory usage can grow indefinitely based on the size of the data.
Both current formats are proprietary, the former prone to bad data (What happens when a username has a hash in it?) and the latter not human readable (or queryable by a commonly available tool).

This seems like a case where a simple file-based embedded database would be ideal, e.g. derby or sqlite. However from my research thus far it appears most of the options involve JDBC drivers, which I don't have available (java.sql.* is not implemented on this device).

Is anybody aware of an existing project which would be a good fit?

分享到QQ

分享到微博