用回形针上传非常慢(独角兽)

发布于 2024-10-25 07:08:09 字数 1164 浏览 1 评论 0原文

坐在这里使用一个简单的 Rails 3 应用程序,其中我有一个简单的图库模型,每个图库都有许多图像。图像模型使用回形针和以下选项进行扩展

has_attached_file :local, 
   :styles => {
     :large => "800x800>", 
     :medium => "300x300>", 
     :thumb => "100x100#", 
     :small => "60x60#"
   }

在我的 gallerys_controller 中,我执行了以下操作,以便与 jQuery-File-Upload 插件。从而得到 json 响应。

def add_image
   gallery = Gallery.find params[:id]
   image = gallery.images.new({:local => params[:local]})
   if image.save
     render :json => {:thumb => image.url(:thumb), :original => image.url}
   else  
    render :json => { :result => 'error'}
   end
end

对我来说,这是相当简单的。但问题来了。在 mongrel 下的开发中,任何类型的上传都可以正常工作,每次上传大约需要 500-1000 毫秒。

然而,当我将其投入生产时,我的独角兽工作人员不断超时,并且当它确实发送图像时,一个文件需要 30-55 秒的时间。

我上传的文件大小约为 100k

我已经使用 ipref 对我的 VPS 和我的开发计算机之间的带宽进行了一些测试,得到了大约 77kbps 的平均速度,因此上传应该不成问题。

请注意,我还使用具有头像的用户模型的同一应用程序进行了非 ajax 文件上传测试。 发展=>在 693 毫秒内完成 302 生产=> Completed 302 Found in 21618ms

有人遇到过类似的(rails3、unicorn)文件上传问题吗?

Sitting here with a simple rails 3 app in which I have a simple Gallery model and each gallery has many images. The image model is extended with paperclip and with the following options

has_attached_file :local, 
   :styles => {
     :large => "800x800>", 
     :medium => "300x300>", 
     :thumb => "100x100#", 
     :small => "60x60#"
   }

In my galleries_controller I have the following action that is implemented in order to work with the jQuery-File-Upload plugin. thereby the json response.

def add_image
   gallery = Gallery.find params[:id]
   image = gallery.images.new({:local => params[:local]})
   if image.save
     render :json => {:thumb => image.url(:thumb), :original => image.url}
   else  
    render :json => { :result => 'error'}
   end
end

To me this is fairly straight forward. But here comes the issue. In Development under mongrel any kind of upload works just fine with about 500-1000ms/upload.

However when I push it in to production I constantly get timeouts of my unicorn workers and when it does send an image through it takes anywhere from 30-55 seconds for one file.

the files I upload are around 100k in size

I have done some testing of the bandwidth between my VPS and my dev computer witH ipref and got an average speed of about 77kbps so the upload should not be a problem.

Note I also did a test with a non ajax file upload using the same app with user model that has an avatar.
Development => Completed 302 Found in 693ms
Production => Completed 302 Found in 21618ms

Anyone experienced a similar issue with (rails3, unicorn) file uploads?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

○愚か者の日 2024-11-01 07:08:09

因此,在深入研究之后,我设法确定在我的 VPS 上,是 ImageMagick 中的 OpenMP 选项导致了运行速度非常慢。所以我的第一次尝试是重建本机 Ubuntu 10.04 软件包并添加 --disable-openmp 标志。由于某种原因失败了,虽然我不确定为什么该软件包在 openMP 仍然处于活动状态的情况下不断出现。我当前的解决方案是从 Ubuntu 10.10 向后移植 ImageMagick。下面是我采取的步骤:

第 1 步下载以下文件:

  • imagemagick_6.6.2.6-1ubuntu1.1.dsc
  • imagemagick_6.6.2.6.orig.tar.bz2
  • imagemagick_6.6.2.6-1ubuntu1 .1.debian.tar.bz2

来自此处

第 2 步解压包

$ dpkg-source -x imagemagick_6.6.2.6-1ubuntu1.1.dsc

第3步编辑规则

$ cd imagemagick-6.6.2.6
$ vim debian/rules

将以下行添加到第25-39行的./configure语句中。我在第 34 行添加了我的。

34: --disable-openmp \

第 4 步添加依赖项并构建(我需要这些依赖项)

$ sudo apt-get install liblqr-1-0-dev librsvg2-dev
$ dpkg-buildpackage -b

第 5 步淘汰旧的,加入新的

$ sudo apt-get remove --purge imagemagick
$ sudo dpkg -i libmagickcore3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i libmagickwand3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i imagemagick_6.6.2.6-1ubuntu1.1_amd64.deb

第 6 步< /strong> 再次实现快速图像转换

_before_ (with openmp)
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real    0m11.602s
user    0m11.414s
sys  0m0.069s

_after_
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real    0m0.077s
user    0m0.058s
sys  0m0.019s

So after digging around I managed to determine that on my VPS it was the OpenMP Option in ImageMagick that was causing the very slow operation. So my first attempt was to rebuild the native Ubuntu 10.04 package with the --disable-openmp flag added. This failed for some reason and while I am not sure why the package kept comming out with openMP still active. My current solution is now instead to backport ImageMagick from Ubuntu 10.10. Below follows the steps I took:

Step 1 download the following files:

  • imagemagick_6.6.2.6-1ubuntu1.1.dsc
  • imagemagick_6.6.2.6.orig.tar.bz2
  • imagemagick_6.6.2.6-1ubuntu1.1.debian.tar.bz2

from here

Step 2 unpack the package

$ dpkg-source -x imagemagick_6.6.2.6-1ubuntu1.1.dsc

Step 3 edit the rules

$ cd imagemagick-6.6.2.6
$ vim debian/rules

Add the the follwing line to the ./configure statment on line 25-39. I added mine on line 34.

34: --disable-openmp \

Step 4 add dependencies and build ( I needed these dependencies)

$ sudo apt-get install liblqr-1-0-dev librsvg2-dev
$ dpkg-buildpackage -b

Step 5 Out with the old, in with the new

$ sudo apt-get remove --purge imagemagick
$ sudo dpkg -i libmagickcore3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i libmagickwand3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i imagemagick_6.6.2.6-1ubuntu1.1_amd64.deb

Step 6 Once again have fast image conversions

_before_ (with openmp)
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real    0m11.602s
user    0m11.414s
sys  0m0.069s

_after_
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real    0m0.077s
user    0m0.058s
sys  0m0.019s
柏林苍穹下 2024-11-01 07:08:09

如果处理需要很长时间,请考虑在单独的工作程序中处理缩略图。

请求:接受文件;将其保存到磁盘;将作业发布到队列
Worker:从队列中弹出作业;创建缩略图; Repeat

Delayed::Job 和 Resque 是很好的解决方案。

If processing takes a long time, consider processing the thumbnails in a separate workers.

Request: accept file; save it to disk; post job to queue
Worker: pop job from queue; create thumbnails; repeat

Delayed::Job and Resque are great solutions for this.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文