我正在使用 Dragonfly 为我的 Rails 应用程序提供处理后的图像。 Dragonfly 依赖 Rack::Cache 来将来访问这些处理过的图像,因此 Dragonfly 不必一次又一次地处理这些图像,从而浪费 CPU 时间。
我的问题从这里开始:如果我是对的,通过 Rack::Cache 发送文件仍然会占用 Rails 进程,那么查看包含 30 个图像的页面,即使这些图像的文件大小很小,也会相当占用 Rails 进程迅速地。如果更多的访问者来查看该页面,那么他们的响应时间将会非常慢。如何通过 X-Sendfile 获取这些文件?
我在 生产.rb
中设置了以下内容,但我知道这些是来自 Rails 的资源,而不是 Dragonfly 文件:
config.serve_static_assets = false
config.action_dispatch.x_sendfile_header = "X-Sendfile"
我知道 Rack::Cache 以某种方式支持 X-Sendfile (可能通过 < a href="https://github.com/rack/rack/blob/master/lib/rack/sendfile.rb" rel="nofollow">Rack::Sendfile) 因为它 生成响应 #to_path
的正文。但是,我不知道如何启用此功能。当我检查来自 Rack::Cache 的文件时,我没有看到任何 X-Sendfile 信息:
Date: Wed, 02 Nov 2011 11:38:28 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.9
Content-Disposition: filename="2.JPG"
Cache-Control: public, max-age=31536000
Etag: "3174d486e4df2e78a5ff9174cacbede5787d4660"
X-Content-Digest: c174408eda6e689998a40db0aef4cdd2aedb3b6c
Age: 28315
X-Rack-Cache: fresh
Content-Length: 22377
Status: 200
Content-Type: image/jpeg
我知道,基于 网上的帖子,我应该会看到类似的内容:
X-Sendfile: /path/to/file
最后我不知道我必须配置它的是 Dragonfly 还是 Rack::Cache(或两者)。 如何让 Dragonfly 和/或 Rack::Cache 通过 X-Sendfile 提供文件?
关于我的设置的信息:
- Rails 3.1.1
- Passenger 3.0.9
- CentOS
- 据我所知, Sendfile 模块已安装。我在虚拟主机配置中指定了
XSendFile On
和 XSendFilePath /path/to/app
,并且 Apache 不会抱怨指令 XSendFile
not现存的。
谢谢!
2011 年 11 月 6 日更新
基于此旧更新,只要将Rack::Sendfile
放在Rack::Cache
前面,就会使用X-Sendfile。我这样做了,这就是我的中间件的样子。但是,这些文件仍然没有 X-Sendfile 标签。同样,我不知道这是否是确定是否启用 X-Sendfile 的可靠方法,因此我检查了乘客队列。当我访问一个页面时,队列似乎受到很大的阻碍。
2011 年 11 月 7 日更新
看来这纯粹是 Rack::Cache 和 Rails 3.1 的问题。虽然 Rack::Cache 支持通过 Rack::Sendfile 使用 X-Sendfile(就像我上面提到的,Rack::Cache,当使用 磁盘 EntityStore,因为从 to_path href="https://github.com/rtomayko/rack-cache/blob/master/lib/rack/cache/entitystore.rb#L105" rel="nofollow">它返回的主体是 File 的子类),Rails 3.1使用自己的存储解决方案。 Rails 3.1 使用 ActiveSupport: :Cache::FileStore,如果您未在 production.rb
文件中指定任何内容,则默认设置。
FileStore 的问题在于它返回的正文是要发送到上游的响应的一部分,因为该正文不响应 to_path
。主体是 ActiveSupport 的实例::缓存::条目。您可以在此处查看< /a> 当要求 FileStore 读取缓存文件时,它会通过 File.open('/path/to/file') {|f| 读取该文件。 Marshal.load(f) } 返回 Entry 的实例。最终传递到上游并返回给客户端的值是 条目#value。
我的问题
为了帮助我决定是否应该修补此问题,或者让 Rails 使用 Rack::Cache 自己的磁盘存储,我有一些问题:
- Rack::Cache 自己的存储解决方案没有使用的原因是什么?未用于 Rails 3.1?为什么 Rails 有自己的?
- 使用元帅有什么原因吗?是否有理由应该发回数据字节流?
我比平时更深入,如果我正确理解了事情,我会感到惊讶。我希望能找到答案!
I'm using Dragonfly to serve processed images for my Rails app. Dragonfly relies on Rack::Cache for future visits to those processed images, so that Dragonfly won't have to process those images again and again, thus wasting CPU time.
My problem starts here: if I'm right that sending a file via Rack::Cache still busies a Rails process, then viewing a page of 30 images, even if these images have a small file size, will tie up the Rails processes pretty quickly. If a couple more visitors come to see that page, then they will experience very slow response times. How do I get these files served via X-Sendfile?
I've set the following in production.rb
, but I know these are for the assets from Rails, not the Dragonfly files:
config.serve_static_assets = false
config.action_dispatch.x_sendfile_header = "X-Sendfile"
I know that Rack::Cache somehow supports X-Sendfile (probably through Rack::Sendfile) because it produces a body that responds to #to_path
. However, I don't know how to enable this. When I check files that come from Rack::Cache, I don't see any X-Sendfile information:
Date: Wed, 02 Nov 2011 11:38:28 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.9
Content-Disposition: filename="2.JPG"
Cache-Control: public, max-age=31536000
Etag: "3174d486e4df2e78a5ff9174cacbede5787d4660"
X-Content-Digest: c174408eda6e689998a40db0aef4cdd2aedb3b6c
Age: 28315
X-Rack-Cache: fresh
Content-Length: 22377
Status: 200
Content-Type: image/jpeg
I know, based on posts around the net, that I'm supposed to see something like:
X-Sendfile: /path/to/file
In the end I don't know if its Dragonfly or Rack::Cache (or both) that I have to configure. How do I get either Dragonfly and/or Rack::Cache to serve files via X-Sendfile?
Info about my setup:
- Rails 3.1.1
- Passenger 3.0.9
- CentOS
- Sendfile module is installed, as far as I know. I have
XSendFile On
and XSendFilePath /path/to/app
specified in my virtualhost configuration, and Apache doesn't complain about the directive XSendFile
not existing.
Thanks!
UPDATE Nov 6, 2011
Based on this old update, as long as Rack::Sendfile
is placed in front of Rack::Cache
, then X-Sendfile will be used. I did that, and this is how my middleware looks like. The files, however, still don't have the X-Sendfile tag. Again, I don't know if that is a sure-fire way of determining if X-Sendfile is enabled, so I checked the Passenger queue. It seems that the queue is greatly encumbered when I visit a page.
UPDATE Nov 7, 2011
It seems this is purely a Rack::Cache and Rails 3.1 issue. While Rack::Cache supports the use of X-Sendfile through Rack::Sendfile (like I mentioned above, Rack::Cache, when using the Disk EntityStore since that responds_to to_path
since the body it returns is a subclass of File), Rails 3.1 uses its own storage solution. Rails 3.1 uses ActiveSupport::Cache::FileStore, which is set by default, if you don't specify anything in your production.rb
file.
The problem with FileStore is that the body it returns, to be part of the response to be sent upstream, because that body doesn't respond to to_path
. The body is an instance of ActiveSupport::Cache::Entry. You can see here that when the FileStore is asked to read a cached file, it reads it via File.open('/path/to/file') {|f| Marshal.load(f) }
which returns an instance of Entry. The value that ultimately gets passed upstream and back to the client, is Entry#value.
My questions
To help me decide whether I should patch this, or to get Rails to use Rack::Cache's own Disk store instead, I have some questions:
- What's the reason Rack::Cache's own storage solutions weren't used for Rails 3.1? Why does Rails have its own?
- Is there a reason Marshal is used? Is there a reason that a bytestream of data should be sent back instead?
I got in deeper than I usually go, and will be surprised if I understood things correctly. I hope to find an answer!
发布评论
评论(2)
作为 Varnish 的替代方案,您可以使用 Apache 的 mod_disk_cache。由于您已经在运行 Apache,因此设置工作会更少。
As an alternative to Varnish, you can use Apache's mod_disk_cache. It would be less work to set up as you are already running Apache.
我最终让它工作起来,尽管使用了 nginx 和 nginx。独角兽而不是阿帕奇和乘客。
正如您在 您的 Github 问题中指出的,您可以将 Rack::Cache 切换回来使用它的标准 file:/ store 而不是rails:/ store,这将允许响应响应
to_path
。Dragonfly 在开发中这样做,如果您愿意,您仍然可以在生产中这样做。这样做需要注意的是,如果您使用任何使用 Rack::Cache 的 Rails 缓存功能,缓存条目将存储在该存储中而不是标准 Rails 存储中,因此如果需要,您需要考虑这一点手动清除任何这些条目。
然后,您还需要确保使用 config.action_dispatch.x_sendfile_header 参数将 Rack::Sendfile 中间件插入到堆栈的前面。如果没有配置参数,Rack::Sendfile 将不会添加标头。
My Gist 显示了 production.rb 中的相关行和我的 nginx 模板。应该很容易适应与 Apache X-Sendfile 模块一起使用。
如果您正在测试此功能,需要注意的另一件事是,例如,如果您仅通过 cURL 发送 HEAD 请求,则您将不会在响应中获得相关的 X-Sendfile 标头,因为 Rack::Cache 不会实际发送正文HEAD 请求,因此 Rack::Sendfile 没有任何可调用
to_path
的内容。I ended up getting this to work, albeit with nginx & unicorn rather than Apache & Passenger.
As you pointed out in your Github issue, you can switch Rack::Cache back to use it's standard file:/ store rather than the rails:/ store, which will allow the responses to respond to
to_path
.Dragonfly does this in development, and you can still do it in production if you like. The caveat with doing this is if you use any of the Rails caching features that use Rack::Cache, the cache entries will be stored in that store rather than the standard Rails one so you'll need to account for that if you need to clear any of those entries manually.
You then also need to make sure that you insert the Rack::Sendfile middleware at the front of the stack with the config.action_dispatch.x_sendfile_header argument. Without the config argument, Rack::Sendfile wont add the header.
My Gist shows my relevant lines in production.rb and my nginx template. Should be easily adapted to work with the Apache X-Sendfile module.
One other thing to note if you're testing this, is that if you only send a HEAD request via cURL for example, you will not get the relevant X-Sendfile header in the response as Rack::Cache wont actually send the body for a HEAD request and so Rack::Sendfile has nothing to call
to_path
on.