使用 ImageMagick Convert 进行大图像处理 - 需要更多吞吐量

发布于 11-26 20:15 字数 1830 浏览 0 评论 0原文

我正在将一些较大的图像从多图像(金字塔)tif 转换为 png 格式。最大图像上的“identity -verbose”报告的重要部分如下:

Image: 
  Format: TIFF (Tagged Image File Format)
  Class: DirectClass
  Geometry: 72224x64080+0+0
  Resolution: 72x72
  Print size: 1003.11x890
  Units: PixelsPerInch
  Type: TrueColor
  Base type: TrueColor
  Endianess: MSB
  Colorspace: RGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
    ...
  Page geometry: 72224x64080+0+0
    ...
  Scene: 2 of 12
  Compression: JPEG
  Orientation: TopLeft
  Properties:
    ...
  Filesize: 1.389GBB
  Number pixels: 4.6281GB
  Pixels per second: 5.516MB
  User time: 218.277u
  Elapsed time: 13:60.020
  Version: ImageMagick 6.7.1-0 2011-07-06 Q16 http://www.imagemagick.org

我打算使用 deepzoom 作曲家使用该图像为 Silverlight 多尺度图像控件生成输入。我的问题是如何在使用 ImageMagick 处理这些图像时使我的系统崩溃 - 转换它们需要很长时间。我看了几篇文章,但似乎没有任何收获。

一些系统和其他相关信息:

 OS: Windows 7 64 bit.
 CPU: Intel Core2 Duo E7300 @ 2.66, 2.67
 RAM: 4.0 GB
 PAGEFILE: 8-12GB on non-OS disk
 "MAGICK_TMPDIR": Yet another empty, non-os disk with 140GB available.

这是“identify -list资源”的结果:

 File         Area       Memory          Map         Disk    Thread
 ------------------------------------------------------------------
 1536     4.1582GB    15.491GiB    30.981GiB    unlimited         2

我正在运行此命令来提取上面引用的图像:

convert "myFN.tif[2]" -limit file 8192GB -limit thread 32 "myFN%d.png"

添加两个限制值似乎没有什么区别。当我运行这个程序时,我的平均 CPU 利用率约为 10%,页面文件提交大小为 3BG。我几乎看不出它正在运行。

Q1) 我还能做些什么来让 ImageMagick 使用更多系统资源吗?我发现的大多数“大图像”链接都在问相反的问题。

Q2) 更改位于此处的“policy.xml”值(例如文件): C:\Program Files\ImageMagick-6.7.1-Q16\www\source 似乎没有影响任何东西 - 更改没有出现在下一个“识别列表资源”中。这有什么技巧吗?

Q3)对于这项任务还有其他提示或想法吗?

谢谢, 大卫

I am converting some largish images from a multi-image (pyramidal) tif to png format. The salient parts of the report from "identity -verbose" on the largest image are here:

Image: 
  Format: TIFF (Tagged Image File Format)
  Class: DirectClass
  Geometry: 72224x64080+0+0
  Resolution: 72x72
  Print size: 1003.11x890
  Units: PixelsPerInch
  Type: TrueColor
  Base type: TrueColor
  Endianess: MSB
  Colorspace: RGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
    ...
  Page geometry: 72224x64080+0+0
    ...
  Scene: 2 of 12
  Compression: JPEG
  Orientation: TopLeft
  Properties:
    ...
  Filesize: 1.389GBB
  Number pixels: 4.6281GB
  Pixels per second: 5.516MB
  User time: 218.277u
  Elapsed time: 13:60.020
  Version: ImageMagick 6.7.1-0 2011-07-06 Q16 http://www.imagemagick.org

I am intending to use deepzoom composer to produce input for the Silverlight multiscaleimage control with this image. My question is how do I bring my system to its knees while processing these images with ImageMagick - it is taking too long to convert them. I have looked at a few articles, but I can't seem to get anywhere.

Some system and other related information:

 OS: Windows 7 64 bit.
 CPU: Intel Core2 Duo E7300 @ 2.66, 2.67
 RAM: 4.0 GB
 PAGEFILE: 8-12GB on non-OS disk
 "MAGICK_TMPDIR": Yet another empty, non-os disk with 140GB available.

Here is the result of "identify -list resource":

 File         Area       Memory          Map         Disk    Thread
 ------------------------------------------------------------------
 1536     4.1582GB    15.491GiB    30.981GiB    unlimited         2

I am running this command to extract the image referenced above:

convert "myFN.tif[2]" -limit file 8192GB -limit thread 32 "myFN%d.png"

Adding the two limit values did not seem to make a difference. When I run this, I average about 10% CPU utilization and have a pagefile commit size of 3BG. I can barely tell that it is running.

Q1) Is there anything else I can do to get ImageMagick to use more system resources? Most of the "large image" links I have found are asking the opposite question.

Q2) Changing "policy.xml" values (such as files) located here:
C:\Program Files\ImageMagick-6.7.1-Q16\www\source
did not seem to affect anything - the changes did not show up in the next "identify -list resource." Is there a trick to this?

Q3) Any other hints or ideas for this task?

Thanks,
David

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

奢欲2024-12-03 20:15:01

libvips 可以将金字塔tiff直接转换为deepzoom金字塔。它是免费的、速度非常快并且不需要太多内存。

例如,我看到:

$ vipsheader vips-pyr.tif 
vips-pyr.tif: 18008x22764 uchar, 3 bands, srgb, tiffload
$ time vips dzsave vips-pyr.tif x.zip
real    0m9.763s
user    0m19.700s
sys 0m4.644s
peak memory: 180mb

这是一个 20,000 x 20,000 金字塔 tiff 在一台小型笔记本电脑上在 10 秒内转换为 deepzoom。它正在编写一个包含金字塔的 zip 文件,因此您可以立即上传到服务器。内存使用与图像宽度成比例,因此它可以处理非常大的图像 --- 我经常处理 250,000 x 250,000 像素的幻灯片。

文档中有一章介绍 dzsave 。

libvips can convert pyramidal tiff directly into deepzoom pyramids. It's free, very fast and doesn't need much memory.

For example, I see:

$ vipsheader vips-pyr.tif 
vips-pyr.tif: 18008x22764 uchar, 3 bands, srgb, tiffload
$ time vips dzsave vips-pyr.tif x.zip
real    0m9.763s
user    0m19.700s
sys 0m4.644s
peak memory: 180mb

That's a 20,000 x 20,000 pyramidal tiff converted to deepzoom in under 10 seconds on a small laptop. It's writing a zip file containing the pyramid, so you can upload to a server immediately. Memory use scales with image width, so it'll do very large images --- I regularly process 250,000 x 250,000 pixel slides.

There's a chapter in the docs introducing dzsave.

愚人国度2024-12-03 20:15:01

对于您(我的)图像,限制因素是像素缓存的大小,它受到设置“MAGICK_AREA_LIMIT”的限制。默认值 4GB 对于 72224 x 64080 来说不够大 - 这需要至少 4.4GB 的设置 - 尝试“MAGICK_AREA_LIMIT=8GB”。

如果您想控制 ImageMagick 对系统 RAM 和系统页面文件的影响,则可以使用“MAGICK_MEMORY_LIMIT”进行限制。事实上,没有太大必要在那里使用大的限制,因为像素缓存的后备位置是映射的内存文件,它们的效率与系统页面文件的数量级相同。尝试“MAGICK_MEMORY_LIMIT=2GB”,将像素缓存保留在其中(并不是说它无论如何都会去那里 - 它比 12GB 大得多。)

您希望像素缓存转到映射内存,所以尝试“MAGICK_MAP_LIMIT” =100GB”以利用您拥有的空间。内存映射文件最终不会位于系统临时目录中,而是位于“MAGICK_TMPDIR”指定的目录中。

为了获得额外的积分,您还可以尝试 Q8 版本,因为您不需要 16 位颜色通道。您可以预期该版本的磁盘 io 大约为一半。

祝你好运!
大卫

For your (my) image, the limiting factor is the size of the pixel cache, which is limited by the setting "MAGICK_AREA_LIMIT". The default of 4GB is not large enough for 72224 x 64080 - that would require a setting of at least 4.4GB - try "MAGICK_AREA_LIMIT=8GB."

If you want to control the impact that ImageMagick has on system RAM and the system page file, then you can limit that using "MAGICK_MEMORY_LIMIT." In truth, there isn't much need to use a large limit there since the fallback location for the pixel cache is mapped memory files, which are on the same order of magnitude of efficiency as the system page file. Try "MAGICK_MEMORY_LIMIT=2GB", to keep the pixel cache out of there (not that it would go there anyways - it is way bigger than 12GB.)

You want the pixel cache to go to mapped memory, so try "MAGICK_MAP_LIMIT=100GB" to take advantage of that space you have. The memory mapped files will end up, not in the system temp directory, but in the directory specified by "MAGICK_TMPDIR".

For extra credit, you also might experiment with the Q8 version, since you don't need 16 bit color channels. You can expect roughly half the disk io with that version.

Good luck!
David

最冷一天2024-12-03 20:15:01

与 Q16 版本相比,Q8 版本使用一半的磁盘空间和时间来完成转换!另外,如果您最终要将图像分解为图块,您可以使用如下命令一步完成:

convert.exe" "WRL_15_1A.tif[2]" -crop 14409x15396 +repage< /代码>
-scene 0 "temp\WRL_15_1A%d.tif"

  • “[2]”调出第三张图像(具有最高
    分辨率。)
  • -crop 参数为宽度和高度的 1/4
    分别给我们 16 个图块。
  • +repage 将所有图块设置在原点 (0,0)
  • “%d”对文件进行编号,从 # 设置的开始
    “-场景”。

The Q8 version uses half the disk space and time to complete a conversion compared to the Q16 version! Also, if you are going to end up breaking up the image into tiles, you can do that in a single step with a command like:

convert.exe" "WRL_15_1A.tif[2]" -crop 14409x15396 +repage
-scene 0 "temp\WRL_15_1A%d.tif"

  • The "[2]" calls out the third image (the one with the highest
    resolution.)
  • The -crop parameters are 1/4 of the width and height
    respectively, giving us 16 tiles.
  • The +repage sets all of the tiles at origin (0,0)
  • The "%d" numbers the files, starting at the # set by
    "-scene".
旧街凉风2024-12-03 20:15:01

Imagemagick 有一种处理大文件的格式 (mpc)。基本上是用磁盘空间换取内存。转换时会生成两个文件:.mpc.cache,您可以在较小的 .mpc 文件上运行 imagemagick 命令。这些文件可能仅适用于您当前版本的 imagemagick,因此它们不适合存档。

Imagemagick has a format for handling large files (mpc). Basically trading disk space for ram. Two files are generated on convert, .mpc and .cache, and you can run imagemagick commands on the smaller .mpc file. These files may only work on your current build of imagemagick, so they aren't suitable for archive.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文