从 MySQL DB 缓存数据 - 技术和适当的 STL 容器？
我正在设计一个数据缓存系统，一次可以保存大量记录，我需要知道要使用什么 stl 容器以及如何使用它。该应用程序是，我有一个非常大的用户记录数据库 - 当他们登录到我的系统时，我想提取他们的记录并缓存一些数据，例如用户名和几个重要属性。当它们与系统交互时，我更新并访问它们的属性。有几个属性非常不稳定，我这样做是为了避免许多事务对数据库造成“影响”。另外，我很少需要使用数据库进行排序或其他任何事情 - 我使用它就像一个美化的二进制保存文件（这就是为什么我很乐意将记录缓存到内存中......）；对我来说更重要的目标是能够扩展到大量用户。
当用户注销、服务器关闭或定期以循环方式（以防万一......）时，我想将其数据写回数据库。
服务器保留自己的：
vector <UserData *> loggedInUsers;
使用 UserData 保留数据库中的用户名（字符串）和其他属性等内容，以及网络句柄等其他临时数据。
我的第一个问题是，如果我需要在此向量中查找特定用户，最快的方法是什么？是否有不同的 stl 容器可以用来更快地完成此操作？我现在要做的是创建一个迭代器，从loggedInUsers.begin() 启动它并迭代到.end()，检查 *iter->username == "foo" 并在找到时返回。如果用户名位于引导程序的末尾，或者引导程序有 5000 个用户，则这是一个严重的延迟。
我的第二个问题是，如何循环安排这些数据写回数据库？每次准备好向数据库写入一些记录时，我都可以调用一个函数。但我不能将迭代器保存到向量，因为它会变得无效。我想做的是有一个旋转队列，我可以在其中访问队列的头部，将其保存到数据库，然后将其旋转到队列的末尾。这似乎是一个很大的开销......我可以使用什么类型来做得更好？
我的第三个问题是，我正在使用 MySQL 服务器和 libmysqlclient 连接器/C..是否有任何类型的内置缓存可以“免费”解决这个问题，或者是否有完全不同的技术？我愿意接受建议
I am designing a data caching system that could have a very large amount of records held at a time, and I need to know what stl container to use and how to use it. The application is that I have an extremely large DB of records for users - when they log in to my system I want to pull their record and cache some data such as username and several important properties. As they interact with the system, I update and access their properties. Several properties are very volatile and I'm doing this to avoid "banging" on the DB with many transactions. Also, I rarely need to be using the database for sorting or anything - I'm using this just like a glorified binary save file (which is why I am happy to cache records to memory..); a more important goal for me is to be able to scale to huge numbers of users.
When the user logs out, server shuts down, or periodically in round-robin fashion (just in case..), I want to write their data back to the DB.
The server keeps its own:
vector <UserData *> loggedInUsers;
With UserData keeping things like username (string) and other properties from the DB, as well as other temporary data like network handles.
My first Q is, if I need to find a specific user in this vector, what's the fastest way to do that and is there a different stl container I can use to do this faster? What I do now is create an iterator, start it at loggedInUsers.begin() and iterate to .end(), checking *iter->username == "foo" and returning when it's found. If the username is at the end of the vector, or if the vector has 5000 users, this is a significant delay.
My second Q is, how can I round-robin schedule this data to be written back to the DB? I can call a function every time I'm ready to write a few records to the DB. But I can't hold an iterator to the vector, because it will become invalid. What I'd like to do is have a rotating queue where I can access the head of the queue, persist it to the DB, then rotate it to be the end of the queue. That seems like a lot of overhead.. what type could I use to do this better?
My third Q is, I'm using MySQL server and libmysqlclient connector/C.. is there any kind of built in caching that could solve this problem "for free", or is there a different technique altogether? I'm open to suggestions
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
A1。你最好有一张地图，这是一棵可以帮你查找的树。使用映射和（假设您有正确的编译器）或 hash_map （执行相同的操作，但查找机制不同）进行测试。对于不同类型的数据存储工作负载，它们具有不同的性能特征。
A2。一个列表可能更适合你——推到前面，拉到最后。 （也可以使用双端队列，但是如果从迭代器中删除，则无法保留迭代器，而可以使用列表）。 push_back 和 pop_front（反之亦然）将允许您保持滚动缓存数据的队列。
A3。您可以尝试 SQLite，它是一个专为简单的应用程序级数据库存储需求而设计的迷你数据库。它也可以完全在内存中工作。
A1. you're better off with a map, this is a tree that does the lookup for you. Test with a map and (assuming you have the right compiler) or a hash_map (which does the same thing, but the lookup mechanism is different). They have different performance characteristics for different types of data storage workloads.
A2. A list would probably be better for you - push to the front, pull off the end. (a deque could also be used, but you cannot keep an iterator if you erase from it, you can with a list). push_back and pop_front (or vice-versa) will allow you to keep a rolling queue of cached data.
A3. You could try SQLite, which is a mini-database designed for simple application-level db storage needs. It can work entirely in-memory too.
您没有说明您的系统做什么或如何访问它，但这种技术可能无法很好地扩展（因为最终您将耗尽内存，并且您用来查找信息的任何内容都不会像数据库），并且不一定能正确处理并发用户，除非您确保数据可以在它们之间正确共享。
也就是说..您可能最好使用以用户名作为键的地图（http://www.cplusplus.com/reference/stl/map/）。
就将其写回数据库而言，为什么不存储一个单独的结构（队列），以便每次将其写入数据库时都可以清除呢？只要您存储指针，它就不会使用更多内存。这让我想到..而不是使用指针，你应该看看智能指针（例如 boost 的shared_ptr）让你可以传递它们而不用担心所有权。
You don't say what your system does or how it's accessed, but this kind of technique probably won't scale well (because eventually you'll run out of memory and whatever you use to find information won't be as efficient as a database) and won't necessarily handle concurrent users properly, unless you make sure that data can be shared properly between them.
That said.. you might be better off using a map (http://www.cplusplus.com/reference/stl/map/) with the username as the key.
In terms of writing it back to the database, why not store a separate structure (a queue) that you can clear every time you write it to the database? As long as you're storing pointers it won't use much more memory. Which brings me to.. rather than using pointers you should take a look at smart pointers (for example boost's shared_ptr) which let you pass them around without worrying about ownership.