使用 memcached 协议和库与 kestrel 交互
编辑
我已将问题移至顶部。我将留下问题的描述以供搜索帮助以及某人可能需要的任何背景信息。
如果您将 memcached 库与 kestrel 一起使用,如果您在集群中使用 2 个以上服务器并利用可靠的读取功能(或任何其他功能),memcached 哈希算法是否可能总是查找错误的位置?是否必须更改 memcached 库中的哈希算法?我错过了什么吗?有人有任何见解吗?
背景信息
Kestrel 用户夸口说,您可以使用任何 memcached 库连接到 Kestrel 集群,以在队列中弹出和推送项目。仔细想想,似乎是有缺陷的。 Memcached 在集群中工作,无需服务器间通信,因为客户端根据组合哈希算法确定密钥存储在何处或将其存储在何处。
kestrel 文档讨论了 kestrel 如何“基本公平”,因为客户端连接到随机的 kestrel 节点以读取或写入队列。如果您使用 memcached 客户端,您的客户端将始终在同一位置查找队列,因为 memcached 的客户端使用一致哈希算法。显然,如果您的集群中只使用单个 kestrel 服务器,那也没关系,只有一次可以查看的地方。即使您使用多个节点,也可能没问题,因为您正在访问静态队列名称,因此散列算法始终在同一位置查找。
但是,Kestrel 会公开额外的功能,这些功能通过修改您从客户端访问的队列名称进行交互(可靠的读取通过添加 /open 启动,并以 /close 结束)。理论上,这应该会导致客户端始终在错误的位置查找队列,并且永远不会检索队列对象,因为它们一致地写入单个节点,并从 不同节点,一致。
谢谢!
EDIT
I've moved the questions to the top. Ill leave the description of the problem for search help and any background info that someone might want.
If you use a memcached library with kestrel, won't the memcached hashing algorithm possibly always look in the wrong place if you use 2+ servers in the cluster and leverage the reliable read features (or any other features)? Do you have to change the hashing algorithm in the memcached library? Am I missing something? Does anyone have any insight?
BACKGROUND INFO
Kestrel users boast that you can use any memcached library to connect to your kestrel cluster to pop and push items off and on the queue. After thinking about this, it seems flawed. Memcached works in a cluster without inter-server communication because the client determines where the key is stored or where to store it, based on the consisting hashing algorithm.
The kestrel documentation talks about how kestrel is "mostly fair" because the client connects to a random kestrel node to read from or write to the queue. If you use a memcached client, your client would always look in the same spot for the queue, because memcached's client uses a consistent hashing algorithm. Obviously, if you're only using a single kestrel server I your cluster, it doesn't matter, there's only once place to look. Even if you use multiple nodes, it might be okay because you're accessing a static queue name, so the hashing algorithm always looks in the same place.
However, extra features are exposed with kestrel that are interacted with by modifying the queue name that you're accessing from the client (a reliable read is initiated by adding /open, and concluded with /close). This should theoretically cause the client to consistently look in the wrong location for the queue, and would never retrieve a queue object, because they're written to a single node, consistently, and being read from a
Different node, consistently.
Thanks!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
免责声明:我没有与红隼一起工作,但听起来不错,我对你的问题做了一些挖掘。
引用 kestrel 文档:
看起来你是对的,你可能应该创建自己的服务器选择算法(这可以在自定义类中轻松完成),然后连接到一台服务器并使用该服务器进行一项操作( set/get/open+close/monitor+confirm),然后根据需要切换到另一项。
如果 memcached 客户端将服务器切换为 /open 和 /close,这确实会搞砸您的事务,因此它并不真正安全,您必须确保在 kestrel 操作中使用相同的服务器,而不是 memcached 操作。
如果您想使用监视器并确认,无论如何您都必须编写自己的客户端类,因此 memcached 客户端库的剂量并不重要。
Disclaimer: I didn't work with kestrel but it sounds good and I did some digging around for your question.
Quoting from the kestrel docs:
It looks like you are right, you should probably just create your own server picking algorithm (this can easily be done in an custom class) and just connect to one server and use that one for one operation(set/get/open+close/monitor+confirm) then switch to another one if you want.
If the memcached client would switch server for a /open and /close this would really screw up your transaction so it's not really safe, you have to make sure you use the same server inside a kestrel operation and not a memcached one.
If you want to use monitor and confirm you will have to write your own client class anyway so it doesn't really matter what the memcached client library dose.