需要一种好方法来存储大量数据
我以每 16.66 毫秒 4096 字节的速率生成数据。这些数据需要持续存储并且将被随机读取。将其放在关系数据库中会很好，但我认为进行如此多的插入会给我正在使用的处理器（ARM11）带来太多开销。而且我不需要 SQLite 等提供的所有功能。
事实上，将这些东西写入文件似乎很诱人，因为虽然大多数时候我只是写入大量数据，但当我确实需要读取数据时，我可以只查找我需要的块。然而，我只知道一路上我会遇到一些问题。特别是当我让这个东西运行一天并最终得到千兆字节的数据时。
这似乎是解决我的问题的一个非常幼稚的解决方案，我需要其他人告诉我，这样我才能开始考虑更好的解决方案。谢谢。
I'm generating data at a rate of 4096 bytes every 16.66ms. This data needs to be stored constantly and will be read randomly. It would be nice to have it in a relational database, but I think doing so many inserts would create too much overhead for the processor I'm working with (ARM11). And I don't need all the features that something like SQLite offers.
In fact, just writing this stuff to a file seems tempting because while most of the time I'll just be writing lots of data, when I actually do need to read data, I can just seek to the block I need. However, I just know I'm going to run into some problem along the way. Especially when I leave this thing running for a day and end up with gigabytes of data.
This just seems like a very naive solution to my problem and I need someone else to tell me so I can start thinking about a better solution. Thanks.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您应该添加更多详细信息以获得更好的答案。您的用例是什么，您是否需要 ACID，您要写入的存储是什么，等等。
您的操作系统是什么，您是否只写入固定大小的记录。只是说我将进行随机访问，这是我的写入速率太不具体了。
您的写入速度为 240 kb/s，即 20 GB/天。
如果您只有固定大小的记录，仅附加数据并使用 Linux，那么纯文件就很好。如果您的存储足够快，也许可以考虑使用一些 fsync 调用。
You should add some more details to get better answers. What are you use cases, do you need ACID, what is your storage you are writing to, etc.
What is your OS, do you only write fixed size records. Just saying like I will do random access and this is my write rate is something is much too unspecific.
You are writing at 240 kb/s, which are 20 GB/day.
If you have just fixed size records, only append data and use Linux, then a plain file is great. Perhaps think about using some fsync calls, if your storage is fast enough.