GAE——分片属性查询
我了解 Google App Engine 中的分片值理论,如下所述:
http://code .google.com/appengine/articles/sharding_counters.html
但是,当我想对已分片的值运行查询时会发生什么情况?我不能简单地查询该值,因为它被随机分配在 N 个不同的计数器中。解决方案是否只是偶尔汇总这些值以更新我的主要实体?我很好奇其他人针对这个问题提出了什么解决方案。
编辑:我刚刚发现了任务队列 API,看起来它可能是在后台更新主值的解决方案。有人尝试过将其与分片并行使用吗?
I understand the theory of sharding values in Google App Engine,as outlined here:
http://code.google.com/appengine/articles/sharding_counters.html
but what happens when I want to run a query on a value that I've sharded? I can't simply query against the value, because it's been split up randomly amongst N different counters. Is the solution just to sum these values back up occasionally to update my main entity? I'm curious to see what solutions others have come up with to this problem.
EDIT: I just discovered the Task Queue API, and it looks like it might be a solution to updating the main value in the background. Anyone tried using this in parallel with sharding?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
你是对的,你不能一次性使用另一个数据存储查询中的总和,因为它是在分片之间分割的。但是,您可以运行初始查询来收集所有分片,将它们在内存中求和,然后使用该总和运行原始查询。
除此之外,是的,任务队列绝对是在后台执行此类工作的好方法。请观看此演讲以获取想法:
http://www.google.com/events/io/2010/sessions/high-throughput-data-pipelines-appengine.html
you're right, you can't use the total sum in another datastore query in a single shot, since it's split between the shards. however, you can run an initial query to gather all of the shards, sum them in memory, and then run your original query using that sum.
beyond that, yes, the task queue is definitely a good approach to doing work like this in the background. take a look at this talk for ideas:
http://www.google.com/events/io/2010/sessions/high-throughput-data-pipelines-appengine.html