从 Memcache 或 blobstore 提供服务更快/更高效/更便宜吗?
我有文件数据(特别是语言资源文件)。这些文件是使用机器翻译 api (goog translate) 自动生成的。它们的更改相对较少,但当主文件发生更改(添加或更改新字符串)时,这会导致所有其他语言文件自动更新。
我正在尝试决定是直接从 blobstore 提供这些文件还是从 memcache 提供这些文件并将它们存储在数据存储中。
哪个更快/更高效?
I have file data (specifically language resource files). These files are automatically generated using machine translation api's (goog translate). They change relatively infrequently but when the master one changes (new string added or changed), this causes all the other language files to be updated automatically.
I'm trying to decide between serving these files directly from the blobstore or serving them from memcache and storing them in the datastore.
Which is faster/more efficient?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
Nick Johnson 在本文中描述了速度权衡。 Blobstore 最擅长处理用户的上传。对于您的问题,您可能会使用数据存储支持的内存缓存获得最快且最便宜的性能。在 Python 中,NDB 将为您自动执行此操作。在 Java 中,使用 objectify。
Nick Johnson described the speed tradeoffs in this article. The blobstore is best at handling uploads from users. For your problem, you will probably get the fastest and cheapest performance using the memcache backed by the datastore. In python, NDB will automate this for you. In java, use objectify.
这实际上取决于您所提供的服务。当人们谈论 blobstore 时,他们通常谈论的是不适合内存缓存的大数据(媒体文件)。我们的应用程序提供大量音频文件,我发现 blobstore 特别适合此,因为它支持渐进式 http 下载。
在这两种情况下,查找时间几乎都是瞬时的(它们都只是映射,您可以通过键查找数据)。送达时间取决于退回的商品。我想不出为什么我会从 blobstore 中取出一些东西并将其放入内存缓存中。这真的不会节省任何时间。
现在数据存储是一个不同的野兽......
It really depends on what you're serving. When people talk about the blobstore they are generally talking about large data (media files) that aren't going to fit in memcache. Our app serves up a lot of audio files and I've found that the blobstore is particularly good for this because it supports progressive-http download.
In both cases the lookup time is virtually instantaneous (they are both just maps and you look up data by a key). The time it takes to serve it depends on the item being returned. I can't think of any reason why I would take something from the blobstore and put it in memcache. It's really not going to save any time.
Now the datastore is a different beast...
每个“哪个更快”问题的答案是“对其进行基准测试”。您的设置的特殊性(磁盘速度、内存访问延迟、带宽、恶魔侵扰)充其量只能给出有关性能机会的一般性答案。事实上,您在 Google App Engine 中运行只会让这变得更加困难 - 您不知道您将获得什么硬件!所以测试一下。
也就是说,像 Google 提供的本地内存缓存可能比任何可能涉及到磁盘的缓存都要快。内存访问延迟比磁盘访问延迟快一个数量级,内存带宽甚至是当今市场上最快的 SSD 的一百倍或更多。
因此,如果您负担得起 RAM 并且希望最大限度地提高响应能力,那么将数据存储在内存中通常会更有效。
The answer to every "which is faster" question is "benchmark it". The particularities of your setup (disk speed, memory access latency, bandwidth, demonic infestations) make any general answer about performance chancy at best. The fact that you're running in Google App Engine just makes this even harder - you don't know what hardware you're going to get! So test it.
That said, it is likely that a local(ish) memcache like Google provides will be faster than anything that might involve hitting the disk. Memory access latency is an order of magnitude faster than disk access latency, and memory bandwidth is a hundred times or more that of even the fastest SSDs on the market today.
So, if you can afford the RAM and you want to maximize your responsiveness, storing your data in memory is generally more efficient.