一般：我希望我要描述的用例是光流问题的一个简单案例，并且由于我对这个主题没有太多了解，我想知道是否任何人对我如何解决我的问题有任何建议。
我已经完成的研究：我已经开始阅读基于翘曲理论的高精度光流估计论文，我计划查看粒子视频论文。我找到了光流的 MATLAB 高精度光流 实现。然而，论文（和代码）似乎描述了非常复杂的概念，可能需要我大量的时间来深入和理解。我希望我的问题的解决方案可能更简单。
问题：我有一系列图像。这些图像描绘了材料破裂的过程，其中材料和背景为黑色，裂纹为白色。我感兴趣的是反向遍历图像序列，试图将破损过程中形成的所有裂纹映射到第一个黑色图像。你可以把这些材料想象成一个大拼图，我试图按照它们破碎的相反顺序将碎片重新组装在一起。
在每个图像中，可能存在一些刚刚出现的裂缝和/或一些已经完全形成的裂缝（从而创建碎片）。在整个破碎过程中，一些碎片可能会分离并进一步破碎。片段也可以彼此远离（后续帧之间的变化很小）。
所需输出：序列中的所有裂纹/线条都映射到序列中的第一个图像。
附加说明：图像有灰度格式（即原始图像）和二进制格式，其中裂缝以白色勾勒，背景为全黑。请参阅下面的一些图像示例。
顶行显示原始图像，底行显示二进制图像。正如您所看到的，随着图像序列的进展，中间的裂缝变得越来越宽。因此，底部裂纹与下部碎片一起移动。当反向遍历序列时，我希望通过算法实现中间裂缝作为一个整体聚集在一起（并将其正确映射到第一张图像），并正确映射底部裂缝，保持其与底部裂缝的正确对应关系（大小和位置）底部片段。
一个序列通常包含大约 30~40 个图像，因此我只显示了开始的子集。此外，虽然这些图像没有显示它，但特定图像可能仅包含裂缝的开始（即其初始外观），并且在后续图像中它变得越来越长并且可能与其他裂缝结合在一起。
语言：虽然没有必要，但我想使用 MATLAB 来实现该解决方案（只是因为与该项目相关的大多数其他代码都是在 MATLAB 中完成的）。然而，如果 OpenCV 可能更容易，我可以灵活地使用我的语言/库。
任何想法都将不胜感激。
General: I'm hoping that the use-case I'm about to describe is a simple case of an optical flow problem and since I don't have much knowledge on the subject, I was wondering if anyone has any suggestions on how I can approach solving my problem.
Research I've already done: I have began reading the High Accuracy Optical Flow Estimation Based on a Theory for Warping paper and am planning on looking over the Particle Video paper. I have found a MATLAB High Accuracy Optical Flow implementation of optical flow. However, the papers (and the code) seem to describe concepts that are very involved and may require a lot of time for me to dig in and understand. I am hoping that the solution to my problem may be more simple.
Problem: I have a sequence of images. The images depict a material breakage process, where the material and background are black and the cracks are white. I am interested in traversing the sequence of images in reverse in an attempt to map all of the cracks that have formed in the breakage process to the first black image. You can think of the material as a large puzzle and I am trying to put the pieces back together in the reverse order that they broke.
In each image, there can be some cracks that are just emerging and/or some cracks that have been fully formed (and thus created a fragment). Throughout the breakage process, some fragments may separate and break further. The fragments can also move farther away from one another (the change is slight between subsequent frames).
Desired Output: All of the cracks/lines in the sequence mapped to the first image in the sequence.
Additional Notes: Images are available in grayscale format (i.e. original) as well as in binary format, where the cracks have been outlined in white and the background is completely black. See below for some image examples.
The top row shows the original images and the bottom row shows the binary images. As you can see, the crack that goes down the middle grows wider and wider as the image sequence progresses. Thus, the bottom crack moves together with the lower fragment. When traversing the sequence in reverse, I hope to algorithmically realize that the middle crack comes together as one (and map it correctly to the first image), and also map the bottom crack correctly, keeping its correct correspondence (size and position) with the bottom fragment.
A sequence typically contains about 30~40 images, so I've just shown the beginning subset. Also, although these images don't show it, it is possible that a particular image only contains the beginning of the crack (i.e. its initial appearance) and in subsequent images it gets longer and longer and may join with other cracks.
Language: Although not necessary, I would like to implement the solution using MATLAB (just because most of the other code that relates to the project has been done in MATLAB). However, if OpenCV may be easier, I am flexible in my language/library usage.
Any ideas are greatly appreciated.
发布评论
评论（3）
向前遍历而不是反向遍历，并且不使用光流。使用断裂线分割黑色部分，随着时间的推移跟踪每个黑色部分的质心。每当出现一条新的断裂线穿过黑色片段时，将该片段分成两部分并继续分别跟踪每个片段。
由此，您应该能够构建一个树结构，表示黑色部分随时间的分割。断裂线可以作为元数据添加到该树中，或许可以将断裂线分配给它们首次出现的段节点。
Traverse forward rather than reverse, and don't use optical flow. Use the fracture lines to segment the black parts, track the centroid of each black segment over time. Whenever a new fracture line appears that cuts across a black segment, split the segment into two and continue tracking each segment separately.
From this you should be able to construct a tree structure representing the segmentation of the black parts over time. The fracture lines can be added as metadata to this tree, perhaps assigning fracture lines to the segment node in which they first appeared.
我同意达米安。大多数光流方法（例如 HAOF）依赖于强度恒定性控制方程 I(x,t)=I(x+v,t+dt) 的一阶泰勒近似。这意味着解决方案取决于图像导数，其中梯度决定运动矢量幅度和角度，即您需要一定量的纹理。然而，非二值化图像的非常低的纹理就足够了。您可以尝试直方图均衡化来增加输入数据的对比度，但对两个输入图像应用相同的转换非常重要。例如如下：
I agree to damian. Most optical flow methods like the HAOF rely on the first-order taylor approximation of the intensity constancy constrian equation I(x,t)=I(x+v,t+dt). That mean the solution depends on image derivatives where the gradient determine the motion vector magnitude and angle i.e. you need a certain amount of texture. However the very low texture of your non-binarised images could be enough. You could try histogram equalization to increase the contrast of your input data but it is important to apply the same transformation for both input images. e.g. as follows:
我建议你遵循你最初的想法，回溯裂缝。你知道裂缝是什么样子，这样你就可以追踪属于裂缝的所有点。您只需使用光流跟踪器跟踪所有白点，从 Lukas-Kanade 跟踪器开始，看看您得到了什么。高精度光流方法是一种全局方法，而且更通用，我将跟踪图像中的所有像素，试图在各处保持一定的平滑度。 LK 是一种本地方法，仅使用每个点周围的小窗口来进行跟踪。问题是，除了裂缝之外，所有像素都是纯黑色的，因此没有什么可跟踪的，您将花费大量时间来尝试跟踪您无法跟踪且不需要跟踪的东西。
如果线条非常直，您最终可能会遇到所谓的光圈问题，您将得到不准确的结果。您还可以尝试一些基于蛇的形状拟合/变形。
I would advise you to follow your initial idea of backtracking the cracks. Yo kind of know how the cracks look like so you can track all the points that belong to the crack. You just track all the white points with an optical flow tracker, start with Lukas-Kanade tracker and see where you get. The high-accuracy optical flow method is a global one and more general, I'll track all the pixels in the image trying to keep some smoothness everywhere. The LK is a local method that will just use a small window around each point to do the tracking. The problem is that appart from the cracks all the pixels are plain black so nothing to track there, you'll just waist time trying to track something that you can't track and you don't need to track.
If lines are very straight you might end up with what's called the aperture problem and you'll get inaccurate results. You can also try some shape fitting/deformation based on snakes.