如何以分布式方式将 Zend_cache 与 memcached 一起使用?
这个问题的标题可能令人困惑,但问题很简单。
我使用 Zend_Cache 和 memcached 作为后端。我有两个模块,称为“最新文章”和“热门文章”。这两个模块都在每个页面上,并使用类似的查询,例如:
Select * from table where status = 'published' and category = '' order by dateCreated|/popularity\
到目前为止,我的表有 150 万行。我在上一个查询中使用的每个字段都有索引。 我缓存最近的文章1小时,热门的4小时。我有 4 个 Web 服务器(php5/apache2)和 1 个数据库服务器(mysql)。表引擎是innoDB。
问题有时我的缓存在重负载期间过期,这使得我的网站不可用,直到再次缓存这些模块。我可以拥有一个新的 MYSQL 服务器。
但是有没有一种方法可以更智能地处理缓存呢?例如,server1 将尝试刷新缓存,而服务器 2,3 和 4 仍将使用缓存中的相同值。
我可以编写一些代码来做到这一点,但我想知道是否有办法直接使用 Zend_Cache 做到这一点?是否有一种设计模式可以应用于我的问题?
[编辑] 我想要可以扩展到 100 台服务器的东西
The title of this question might be confusing but the problem is simple.
I'm using Zend_Cache with memcached as a backend. I have two module called "Last articles" and "Popular articles". Both of this module are on every pages and use a similar query such as :
Select * from table where status = 'published' and category = '' order by dateCreated|/popularity\
My table have 1.5 million rows so far. I have indexes on every field that I'm using in the previous query.
I cache the recent articles for 1hour and the popular for 4hours. I have 4 web server (php5/apache2) and 1 database server (mysql). The table engine is innoDB.
The problem some time my cache expire right in the middle of a heavy load, which make my web site unavailable until those modules are cached again. I could had a new MYSQL server.
But is there a way to handle the caching in a smarter way? Like for example the server1 will try to refresh the cache while server 2,3 and 4 will still use the same value out of the cache.
I can write some code to do that, but I was wondering if there is way to do that directly with Zend_Cache? Of if there is a design pattern that i could apply to my problem?
[EDIT] I want something that I could scale up to 100 servers
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
为什么不让缓存永不过期,而不是依赖缓存过期,然后在 HTTP 请求期间(或者,更成问题的是,在多个并发请求期间)重新填充缓存呢?
然后安排一些直到脚本来运行昂贵的查询(仅一次!)并在后台更新缓存。
Instead of relying on the cache expiring and then being repopulated during an HTTP request (or, more problematically, during several concurrent requests), why not have the cache never expire?
Then schedule some untility script to run your expensive queries (just once!) and update the cache in the background.
一切皆有可能:)
分布式内存缓存(serv1、2、3、4)。
仅将 serv4 用于 ReCache。
设置“仅限内部”网站(用户不可见)。
删除“将刷新某些类别”的部分。
获取“阅读最多的文章”->解析apache访问日志。
并将 url 重新提交到 server4。
有访问时间,因此您只能获取所需的部分,即 2 到 6 小时前。
分布式 memcache 会自动将其值填充到 serv1、2、3。
everything is possible :)
distributed memcache (serv1,2,3,4).
use serv4 only for ReCache.
set up a "internal only" webSite (not visible for users).
strip the part that "would refresh some categories".
to get "most read articles" -> parse apache access logs.
and re-submit urls to server4.
there is access time, so you can get only the needed part i.e. from 2 to 6 hours ago.
distributed memcache will auto-populate it's values to serv1,2,3.
这是您正在执行的实际查询吗?
也许不要寻找高级缓存解决方案,而是看看为什么这个查询会给您的数据库服务器带来压力。具有 1.5m 行的表并不罕见。
您是否尝试添加 LIMIT 子句或仅选择您需要的列:
它将显着减少数据库和 Web 服务器之间的流量。
Is that the actual query you're executing?
maybe instead of searching for advanced caching solutions, see why this query stresses your database server. A table with 1.5m rows is not something unusual.
Did you try adding a LIMIT clause or select only the columns you require:
It'll reduce the traffic between the database and the web servers significantly.
我终于实现了一个继承自Zend_Cache_Backend_Libmemcached的类
我正在重写 load() 方法。
我的每台服务器都有主机名,并以一组数字结尾,例如 serv01、serv02、serv03、serv04。
主要思想是每个服务器都会认为缓存在不同的时间过期。例如serv01会认为缓存在实际过期前20分钟过期,serv02将是15分钟,serv03是10分钟,serv04是5分钟。
通过这样做,我的缓存将永远不会在每台服务器上同时刷新,并且如果一台服务器关闭,缓存将由另一台服务器刷新。
I finally implemented a class that inherit from Zend_Cache_Backend_Libmemcached
I'm overriding the load() method.
Each of my server has there hostname finishing by a set of number such as serv01, serv02, serv03, serv04.
The main idea is taht each server will think that the cache expired at different time. For example serv01 will think that the cache is expired 20minutes before it actually expires, serv02 will be 15minutes, serv03 10minutes and serv04 5minutes.
By doing so my cache will never be refresh at the same time on each server, and if one server is down the cache will be refresh by another server.