内容感知填充如何工作？
在即将推出的 Photoshop 版本中，有一项称为“内容感知填充”的功能。
此功能将根据周围图像填充图像选区 - 以至于它可以生成灌木丛和云，同时与周围图像无缝连接。
请参阅 http://www.youtube.com/watch?v=NH0aEp1oDOI 进行预览我正在谈论的 Photoshop 功能。
我的问题是： 此功能如何通过算法发挥作用？
In the upcoming version of Photoshop there is a feature called Content-Aware fill.
This feature will fill a selection of an image based on the surrounding image - to the point it can generate bushes and clouds while being seamless with the surrounding image.
See http://www.youtube.com/watch?v=NH0aEp1oDOI for a preview of the Photoshop feature I'm talking about.
My question is:
How does this feature work algorithmically?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
我是前面提到的 PatchMatch 论文的合著者，我与创意技术实验室的 Ivan Cavero Belaunde 和 Eli Shechtman 以及 Photoshop 的 Jeff Chien 一起领导了 Photoshop 中原始内容感知填充功能的开发团队。
Photoshop 的内容感知填充使用 PatchMatch 论文中描述的算法的高度优化的多线程变体，以及称为“SpaceTime Video Completion”的旧方法。此功能的以下技术页面引用了这两篇论文：
http://www.adobe.com/technology/projects/content-aware-fill.html
您可以在 Adobe 研究网页上找到有关我们的更多信息。
I am a co-author of the PatchMatch paper previously mentioned here, and I led the development of the original Content-Aware Fill feature in Photoshop, along with Ivan Cavero Belaunde and Eli Shechtman in the Creative Technologies Lab, and Jeff Chien on the Photoshop team.
Photoshop's Content-Aware Fill uses a highly optimized, multithreaded variation of the algorithm described in the PatchMatch paper, and an older method called "SpaceTime Video Completion." Both papers are cited on the following technology page for this feature:
http://www.adobe.com/technology/projects/content-aware-fill.html
You can find out more about us on the Adobe Research web pages.
我猜测，对于较小的洞，他们会抓住该区域周围类似纹理的补丁来填充它。这是在题为“PatchMatch：结构图像编辑的随机对应算法" 作者：Connelly Barnes 等人，在 SIGGRAPH 2009 中。对于较大的漏洞，他们可以利用大型图片数据库类似的全球统计数据或纹理，如“使用数百万张照片完成场景中所述”。如果他们能以某种方式将两者融合在一起，我认为它应该像视频中那样工作。
I'm guessing that for the smaller holes they are grabbing similarly textured patches surrounding the area to fill it in. This is described in a paper entitled "PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing" by Connelly Barnes and others in SIGGRAPH 2009. For larger holes they can exploit a large database of pictures with similar global statistics or texture, as describe in "Scene Completion Using Millions of Photographs". If they somehow could fused the two together I think it should work like in the video.
GIMP 长期以来都有非常相似的算法。它被称为 resynthesizer ，也许你应该能够找到它的来源（也许在项目网站）
编辑
ubuntu 存储库
也提供了源代码
在这里您可以看到使用 GIMP 处理相同的图像：http://www.youtube .com/watch?v=0AoobQQBeVc&feature=related
There is very similar algorithm for GIMP for a quite long time. It is called resynthesizer and probably you should be able to find a source for it (maybe at the project site)
EDIT
There is also source available at the ubuntu repository
And here you can see processing the same images with GIMP: http://www.youtube.com/watch?v=0AoobQQBeVc&feature=related
好吧，出于显而易见的原因，他们不会透露。这种技术的总称是“修复”，你可以查一下。
具体来说，如果您查看 Criminisi 在 Microsoft http 期间所做的事情： //citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.67.9407 以及 Todor Georgiev 现在在 Adobe http://www.tgeorgiev.net/Inpainting.html，您将能够做出很好的猜测。我想说，90% 的猜测应该足够好了。
Well, they are not going to tell for the obvious reasons. The general name for the technique is "inpainting", you can look this up.
Specifically, if you look at what Criminisi did while in Microsoft http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.67.9407 and what Todor Georgiev does now at Adobe http://www.tgeorgiev.net/Inpainting.html, you'll be able to make a very good guess. A 90% guess, I'd say, which should be good enough.
我正在研究类似的问题。据我了解，他们通常使用“PatchMatch”或“非参数补丁采样”。
PatchMatch：随机对应算法
用于结构图像编辑
I work on a similar problem. From what i read they use "PatchMatch" or "non-parametric patch sampling" in general.
PatchMatch: A Randomized Correspondence Algorithm
for Structural Image Editing
作为猜测（这就是全部），我希望它对图像进行一些频率分析（有些像傅立叶变换）。通过仅查看选区边缘的图像并忽略中间的图像，它可以推断回中间。如果设计师选择了正确的颜色平原以及其他什么，他们应该能够生成在边缘无缝地融入图像的纹理。
编辑：查看视频中的最后一个示例；如果您查看原始图像顶部的任一边缘，您会发现选择线正好沿着云中的“间隙”延伸，而在中间有一个“凹凸”。如果我的猜测正确的话，这些就是我希望看到的文物。 （OTOH，我还希望看到它们，因为它在选择边界上使用了某种 sudo 镜像。）
As a guess (and that's all that it would be) I'd expect that it does some frequency analysis (some like a Fourier transform) of the image. By looking only at the image at the edge of the selection and ignoring the middle, it could then extrapolate back into the middle. If the designers choose the correct color plains and what not, they should be able to generate a texture that seamlessly blends into the image at the edges.
edit: looking at the last example in the video; if you look at the top of the original image on either edge you see that the selection line runs right down a "gap" in the clouds and that right in the middle there is a "bump". These are the kind of artifacts I'd expect to see if my guess is correct. (OTOH, I'd also expect to see them is it was using some kind of sudo-mirroring across the selection boundary.)
一般方法是内容感知填充或接缝雕刻。 Ariel Shamir 的团队负责这里的开创性工作，该工作已在 SIGGRAPH 2007 上展示。请参阅：
http://www.faculty.idc.ac。 il/arik/site/subject-seam-carve.asp
编辑：请参阅内容感知填充合著者的回答。我很快就会删除这个。
The general approach is either content-aware fill or seam-carving. Ariel Shamir's group is responsible for the seminal work here, which was presented in SIGGRAPH 2007. See:
http://www.faculty.idc.ac.il/arik/site/subject-seam-carve.asp
Edit: Please see answer from the co-author of Content-Aware fill. I will be deleting this soon.