Flask/Python/WSGI 与 PHP Apache 共享内存存储(如 apc_store/apc_fetch)有什么好的相似之处?
我已经用 PHP 进行了几年的大型游戏服务器开发。负载均衡器将传入请求委托给集群中的一台服务器。为了提高性能,我们开始使用 apc_store
和 apc_fetch 直接在 Apache 共享内存中缓存该集群中每个实例上的所有静态数据(本质上是游戏世界的模型对象)
。
出于多种原因,我们现在开始使用 Flask 微框架在 Python 中开发类似的游戏框架。乍一看,该实例的内存存储似乎没有直接转换为 Python/Flask。我们目前正在考虑在每个实例上本地运行 Memcached(以避免从我们的主 Memcached 集群通过网络传输相当大的模型对象。)
我们可以使用什么来代替?
I've done a couple of years of large-scale game server development in PHP. A load balancer delegates incoming requests to one server in a cluster. In the name of better performance, we began caching all static data (essentially the game world's model objects) on each of the instances in that cluster, directly in Apache shared memory, using apc_store
and apc_fetch
.
For a number of reasons, we're now beginning to develop a similar game framework in Python, using the Flask microframework. At first glance, this instance's memory store is the one piece that doesn't appear to translate directly to Python/Flask. We're presently considering running Memcached locally on each instance (to avoid streaming fairly large model objects over-the-wire from our main Memcached cluster.)
What can we use instead?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
我认为即使在这种情况下,您也可能需要考虑拥有一个集中式键/值存储系统,而不是在每个服务器上拥有一系列独立的系统。除非您的负载均衡器始终将相同的用户重定向到相同的服务器,否则您可能会遇到这样的情况:用户的请求每次都会路由到不同的服务器,因此每个节点都必须检索游戏状态,而不是从共享缓存访问它。
此外,每个系统上的本地键/值存储可能产生的内存压力可能会减慢游戏服务器的其他功能。尽管这在很大程度上取决于缓存的数据量。
一般来说,最好的方法是运行一些基准测试,看看使用 memcached 集群可以获得什么样的性能,以及与本地存储相比,您要存储的对象类型。
根据您希望从键/值存储中获得哪些其他功能,您可能还想研究一些替代方案,例如 mongodb (http:// /www.mongodb.org/)。
I would think that even in this case you might want to consider having a centralized key/value store system rather than a series of independent ones on each server. Unless your load balancer always redirects the same users to the same servers you could run into a case where a user's requests are routed to different servers each time so each node would have to retrieve the game state instead of accessing it from a shared cache.
Also the memory strain that a local key/value store on each system might incur could slow down your game server's other functions. Though that largely depends on the amount of data being cached.
In general the best approach would be to run some benchmarks to see what kind of performance you'd get with a memcached cluster and the types of objects you're storing vs local storage.
Depending on what other features you want from you key/value store you might also want to look into some alternatives like mongodb (http://www.mongodb.org/).
[五个月后]
我们的游戏框架完成了。
最后,我们决定将静态数据存储在每个 Web 服务器中完全初始化的 sqlalchemy 模型实例中。当新启动的游戏服务器预热时,这些实例首先通过访问共享 MySQL 数据库来构建。
由于我们的模型工厂遵循实例池,因此每个服务器的每次部署只需要构建一次模型实例 - 这很重要,因为在我们的规模下,MySQL 会在任何类型的持续负载下哭泣。我们通过将项目定义尽可能靠近我们的应用程序代码(在应用程序代码本身中)来实现不通过网络传输此数据的目标。
我现在意识到我原来的问题很天真,因为与 LAMP 堆栈不同,Flask 服务器在请求之间保持运行,服务器的内存本身就是“共享内存”——不需要像 APC 这样的东西来实现这一点。事实上,任何东西超出了请求处理范围本身和 Flask 的线程安全本地存储,可以被认为是“共享内存”。
[Five-months later]
Our game framework is done.
In the end, we decided to store the static data in fully initialized sqlalchemy model instances in each web server. When a newly-booted game server is warming up, these instances are first constructed by hitting a shared MySQL db.
Since our Model factories defer to an instance pool, the Model instances need only be constructed once per deployment per server – this is important, because at our scale, MySQL would weep under any sort of ongoing load. We accomplished our goal of not streaming this data over the wire by keeping the item definitions as close to our app code as possible: in the app code itself.
I now realize that my original question was naive, because unlike in the LAMP stack, the Flask server keeps running between requests, the server's memory itself is "shared memory" – there's no need for something like APC to make it so. In fact, anything outside of the request processing scope it self and Flask's threadsafe local store, can be considered "shared memory".