文件系统和 Memcached 哪个缓存更快/更好?

发布于 2024-09-08 00:25:14 字数 44 浏览 2 评论 0原文

我认为我还不清楚,从文件或从 memcached 读取内容更快吗?为什么?

I don't think it's clear to me yet, is it faster to read things from a file or from memcached? Why?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(8

小伙你站住 2024-09-15 00:25:14

Memcached 速度更快,但内存有限。 HDD 很大,但 I/O 速度比内存慢。您应该将最热门的内容放入 memcached,而所有其他内容都可以缓存文件
(或者像这些人一样,投入一些钱来增加内存 :)

对于一些基准测试,请参阅:缓存性能比较(文件、Memcached、查询缓存,APC)

理论上:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http ://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Memcached is faster, but the memory is limited. HDD is huge, but I/O is slow compared to memory. You should put the hottest things to memcached, and all the others can go to cache files.
(Or man up and invest some money into more memory like these guys :)

For some benchmarks see: Cache Performance Comparison (File, Memcached, Query Cache, APC)

In theory:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

慕巷 2024-09-15 00:25:14

有很多不同的方面可能有利于其中之一:

  • 您是否需要/想要在多个服务器之间共享此数据?文件系统是本地的,memcached 通过网络访问。
  • 您缓存的项目有多大?文件系统可能更适合大型对象。
  • 每页可能有多少个 memcached 请求? TCP 连接和拆卸可能比本地计算机上的简单文件系统 stat() 花费更多时间。

我建议您查看您的用例并对这两种方法进行一些分析。如果你可以不用使用文件系统,那么我会的。添加 memcached 会增加另一层复杂性和潜在的故障点(memcached 客户端/服务器)。

值得一提的是,关于磁盘与内存性能的其他评论很可能是学术性的,因为如果定期访问文件系统数据,那么它可能无论如何都会位于操作系统或磁盘缓存中。

There are quite a few different aspects that might favour one or the other:

  • Do you need/want to share this data between multiple servers? The filesystem is local, memcached is accessed over a network.
  • How large are the items your caching? The filesystem is likely to be better for large objects.
  • How many memcached requests might be there per page? TCP connections and tear-downs might take up more time than a simple filesystem stat() on the local machine.

I would suggest you look at your use case and do some profiling of both approaches. If you can get away with using the filesystem then I would. Adding in memcached adds in another layer of complexity and potential points of failure (memcached client/server).

For what it's worth the other comments about disk vs memory performance might well be academic as if the filesystem data is being accessed regularly then it'll likely be sitting in OS or disk cache memory anyway.

A君 2024-09-15 00:25:14

“更快”不能在没有上下文的情况下使用。
例如,由于网络延迟,访问远程服务器上的 memcached 中的数据可能会“变慢”。另一方面,通过 10Gb 网络从远程服务器内存读取数据可能比从本地磁盘读取相同数据“更快”。

在文件系统上缓存和使用 memcached 之间的主要区别在于,memcached 是一个完整的缓存解决方案。所以有 LRU 列表、过期概念(数据新鲜度)、一些高级操作,如 cas/inc/dec/append/prepend/replace。

Memcached 易于部署和监控(我们如何区分文件系统上的“缓存”工作负载和内核?我们可以计算缓存数据的总量吗?数据分布?容量规划?等等)。

还有一些混合系统,例如 cachelot
基本上,它是可以直接嵌入到应用程序中的 memcached,因此无需任何系统调用或网络 IO 即可访问缓存。

"Faster" can not be used without context.
For example, accessing data in memcached on remote server can be "slower" due to network latency. In the other hand, reading data from remote server memory via 10Gb network can be "faster" than reading same data from local disk.

The main difference between caching on the filesystem and using memcached is that memcached is a complete caching solution. So there is LRU lists, expiration concept (data freshness), some high-level operations, like cas/inc/dec/append/prepend/replace.

Memcached is easy to deploy and monitor (how can we distinguish "cache" workload on filesystem from, let's say kernel? Can we calculate total amount of cached data? Data distribution? Capacity planning? And so on).

There are also some hybrid systems, like cachelot
Basically, it's memcached that can be embedded right into the application, so the cache would be accessible without any syscalls or network IO.

橪书 2024-09-15 00:25:14

其实并不是从内存读取比从硬盘读取快很多那么简单。如您所知,Memcached 是基于 tcp 连接的,如果每次想要获取某物或将某物设置到 memcached 服务器时都建立连接(这是大多数程序员所做的),那么它的性能可能比使用文件缓存差。您应该使用静态 Memcached 对象,并重用该对象。其次,现代操作系统会缓存经常使用的文件,这使得文件缓存可能比实际上是 TCP 连接的内存缓存更快。

In fact, it is not as simple as that reading from memory is much faster than reading from HDD. As you known, Memcached is based on tcp connection, if you make connection each time you want to get sth or set sth to memcached server(that is most of programmers do), it proberly performs poorly than using file cache. You should use static Memcached object, and reuse the object. Secondly, the modern OS's will cached files that are frequently used, that makes file caches might be faster than memcaches which are actualy TCP connections.

清秋悲枫 2024-09-15 00:25:14

缓存类型 |缓存获取/秒
数组缓存 | 365000
APC 缓存 | 98000
文件缓存 | 27000
Memcached 缓存 (TCP/IP) | 12200
MySQL 查询缓存 (TCP/IP) | 9900
MySQL 查询缓存(Unix 套接字)| 13500
从表中选择 (TCP/IP) | 5100
从表中选择(Unix 套接字)| 7400

来源:
https://surniaulula.com/os/unix/memcached-vs-disk -cache/

我的来源:)
https://www.percona.com/blog/2006 /08/09/缓存性能比较/

Cache Type | Cache Gets/sec
Array Cache | 365000
APC Cache | 98000
File Cache | 27000
Memcached Cache (TCP/IP) | 12200
MySQL Query Cache (TCP/IP) | 9900
MySQL Query Cache (Unix Socket) | 13500
Selecting from table (TCP/IP) | 5100
Selecting from table (Unix Socket) | 7400

Source:
https://surniaulula.com/os/unix/memcached-vs-disk-cache/

Source of my source :)
https://www.percona.com/blog/2006/08/09/cache-performance-comparison/

调妓 2024-09-15 00:25:14

你对细节太吹毛求疵了。我相信您寻找的答案取决于具体情况。据我所知,很少有事情总是比其他事情更好。

显然,读取文件系统的内容不会更快(假设它是硬盘驱动器)。即使是 SDD 也会比内存读取慢得多。原因是 HDD/文件系统是为了容量而不是速度而构建的,而 DDR 内存因此特别快。

良好的缓存意味着将经常访问的部分保留在内存中,而将不经常访问的部分保留在磁盘上(持久存储)。这样,正常情况下的缓存实现将大大改善。这就是你的目标。确保您充分了解理想的缓存策略。这将需要广泛的基准测试和测试。

You're being awefully vauge on the details. And I believe the answer your looking for depends on the situtation. To my knowledge very few things tend to be better than the other all the time.

Obviously it woudln't be faster to read things of the file system (assuming that it's a harddrive). Even a SDD will be noticably slower than in-memory reads. And the reason for that is that HDD/FileSystem is built for capacity not speed, while DDR memory is particulary fast for that reason.

Good caching means to keep frequently accessed parts in memory and the not so frequently accessed things on disk (persistent storage). That way the normal case would be vastly improved by your caching implementation. That's your goal. Make sure you have a good understanding of your ideal caching policy. That will require extensive benchmarking and testing.

与之呼应 2024-09-15 00:25:14

这取决于缓存是否存储在本地。 Memcache 可以通过网络存储数据,这不一定比本地磁盘更快。

It depends if the cache is stored locally. Memcache can store data across a network, which isn't necessarily faster than a local disk.

知足的幸福 2024-09-15 00:25:14

如果该文件存储在磁盘中,并且经常访问它,那么很有可能在 RAM 中找到它(作为最近访问的文件),或者我在这里丢失了一些东西?
是的,第一次读取将来自磁盘,速度非常慢,但是后续读取呢(假设文件很热并且有大量读取),这应该比 memcached 更快,因为这是纯 RAM 读取

If that file is stored in disk, and it is accessed frequently, there will be a high probability of finding it in the RAM (as a recently accessed file) or I am missing something here?
Yes the first read will be from the disk which is awefully slow, but what about the subsequent reads (assuming that file is hot and its getting lots of reads) this should be even faster than memcached as this is a pure RAM read

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文