缓存数据记录器数据的方法
您好，
我一直在开发一个与数据记录器交互的 C#.NET 应用程序。用户可以查询并获取指定时间段内的日志，并查看数据的图表。通常，每分钟都会创建一个新的数据日志，并存储一些参数的测量结果。为了从记录器中获取有意义的信息，需要获取合理数量的日志 - 至少几天的数据。硬件接口是设备上的 UART 至 USB 模块，它将传输限制为最大约 30 个日志/秒。当读取几天/几周内获取的数据时，这会变得非常慢。
我想做的是提高用户的感知性能。我意识到，由于硬件速度限制，用户至少在第一次获取更大的数据集时必须等待完整的下载周期。我的目标是缓存应用程序看到的所有数据，以便在再次请求时可以更快地获取数据。我一直在考虑的方法是使用轻型数据库，例如 SqlServerCe，它可以在收到数据日志时存储它们。然后我希望在查询设备的日志之前首先搜索缓存。缓存将使用请求获取的尚未缓存的任何日志进行更新。
最后我的问题是——您认为这是一个好方法吗？您能想到更好的替代方案吗？我尝试在 SO 和 Google 上搜索以强化这个想法，但我主要遇到的是有关网络请求/内容缓存的讨论。
感谢您的任何反馈！
Greetings,
I've been working on a C#.NET app that interacts with a data logger. The user can query and obtain logs for a specified time period, and view plots of the data. Typically a new data log is created every minute and stores a measurement for a few parameters. To get meaningful information out of the logger, a reasonable number of logs need to be acquired - data for at least a few days. The hardware interface is a UART to USB module on the device, which restricts transfers to a maximum of about 30 logs/second. This becomes quite slow when reading in the data acquired over a number of days/weeks.
What I would like to do is improve the perceived performance for the user. I realize that with the hardware speed limitation the user will have to wait for the full download cycle at least the first time they acquire a larger set of data. My goal is to cache all data seen by the app, so that it can be obtained faster if ever requested again. The approach I have been considering is to use a light database, like SqlServerCe, that can store the data logs as they are received. I am then hoping to first search the cache prior to querying a device for logs. The cache would be updated with any logs obtained by the request that were not already cached.
Finally my question - would you consider this to be a good approach? Are there any better alternatives you can think of? I've tried to search SO and Google for reinforcement of the idea, but I mostly run into discussions of web request/content caching.
Thanks for any feedback!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
看起来是一个非常合理的方法。就我个人而言，我会使用 SQL CE 进行存储，确保对保存记录日期时间的列进行索引，然后在索引上使用 TableDirect 来获取和插入数据，这样速度会非常快。由于您的数据已经按时间顺序排列，因此无需涉及任何缓慢的 SQL 查询处理器，只需查找日期（或结束时间）并使用 SqlCeResultSet 前滚即可。您最终的速度将仅受 I/O 限制。我分析了在一个项目中所做的非常非常相似的事情，发现使用 SQLCE 的 TableDirect 与平面二进制文件一样快。
Seems like a very reasonable approach. Personally I'd go with SQL CE for storage, make sure you index the column holding the datetime of the record, then use TableDirect on the index for getting and inserting data so it's blazing fast. Since your data is already chronological there's no need to get any slow SQL query processor involved, just seek to the date (or the end) and roll forward with a SqlCeResultSet. You'll end up being speed limited only by I/O. I profiled doing really, really similar stuff on a project and found TableDirect with SQLCE was just as fast as a flat binary file.
我认为您想要以某种可查询的形式在本地存储它是正确的。
我强烈推荐 SQLite。 这里有一个 .NET 类。
I think you're on the right track wanting to store it locally in some queryable form.
I'd strongly recommend SQLite. There's a .NET class here.