opencv 中检测叶状形状的最佳方法
我开始在我正在从事的一个项目中使用 OpenCV。
对于这个项目，我在花盆里种植植物。不同发育阶段的静态图片是从上方拍摄的（见下图）。
目标是检测并最终测量生长植物在不同发育阶段的叶子。< br> 我尝试了一种使用 Canny 边缘检测的简单方法，但效果不太好（见下图），因为它还会检测土壤中的小边缘（无论我使用哪种阈值）。
我认为更好的方法是首先按颜色分割图像，然后使用一些边缘检测算法来检测叶子。
有更好的方法吗？
叶子结构是预先已知的。我可以使用机器学习/分类算法来获得更好的结果吗？
我也没有想过如何测量叶子的大小？是否有测量尺寸和其他描述符的常见模式？ （也许图片中有已知尺寸的参考对象？）。
最后，我还必须在某种程度上处理遮挡问题。这在图片中不可见，但在后期的开发阶段，我可能必须处理重叠的叶子。 有什么方法可以解决这个问题吗？
我不能让图片对我有利（即使土壤变黑等），因为可能有数千种植物需要处理。
总结我的问题：
- 检测土壤中叶子的最佳方法是什么（见图）？
- 机器学习算法可以改善检测吗？
- 如何测量叶子的大小？
- 如何处理遮挡/重叠的叶子？
我真的很感谢一些指点或想法。
更新（基于 Jeff7 评论）：
我首先将均值偏移颜色分割与洪水填充算法一起运行，最终得到了这张图片：
当我现在对该图片运行精明的边缘检测 + findcontours 时，结果要好得多：
I started to play around with OpenCV for a project I am working on.
For this project I am growing plants in pots. Static pictures of different developmental stages are taken from above (see figure below).
The goal is to detect and ultimately measure (i.e. size) the leaves of the growing plants during different developmental stages.
I tried a naive approach of using Canny edge detecting, however it doesn't work that well (see figure below) because it also detects small edges in the soil (no matter what kind of threshold I used).
I think the better approach is to first segment the image by color and then use some edge detection algorithm to detect the leaves.
Is there a better way to do it?
The leaf structure is known beforehand. Could I use machine learning/classification algorithm to obtain even better results?
I also haven't thought about how to measure the size of the leaves? Are there any common patterns for measuring size and other descriptors? (maybe have a reference object with known size in the picture?).
Finally I also have to deal with occlusion to some extend. This is not visible in the pictures but in later development stages I might have to deal with overlapping leaves.
Are there any approaches to deal with that?
I can't bias the picture in my favor (i.e. blacking out the soil, etc) as there might be thousands of plants which have to be processed.
To summarize my questions:
- What's the best approach to detect leafs in a soil (see figures) ?
- Can Machine learning algorithm improve detection?
- How can I measure the size of the leaves?
- How to deal with occlusion/overlapping leaves?
I would be really thankful for some pointers or ideas.
Update (based on Jeff7 comments):
I first ran the mean shift color segmentation together with a floodfill algorithm and ended up with this picture:
When I now run the canny edge detection + findcontours on the that picture the results are much better:
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
由于您对拍摄图像的条件有很大的控制权，因此偏见对您有利。当您想要捕捉图像时，用黑卡制作一个遮罩，将其放置在植物根部周围。您的问题简化为在黑色背景上发现绿色像素的问题之一。由于您还可以根据植物的位置控制相机的位置，因此您应该能够进行安排，使像素/毫米比在一系列图像中保持恒定。叶子面积就是计算像素的简单问题。
这还没有解决遮挡问题。您可能可以通过拍摄另外 2 张图像，即植物在 2 个正交平面中的立面图（再次使用黑色背景）来了解植物的形状。
评论后编辑...
好吧，你的问题包括这样的陈述“对于这个项目，我正在花盆里种植一种特定的植物”，现在你想要处理数千种该死的东西。我仍然会寻求使图像偏向于你的可能性。例如，如果您发布的图像背景中没有绿色塑料，那么您可能会得到一张可以通过对图像的绿色通道进行简单阈值处理而高精度分离的图片。因此，去掉绿色背景，并在成像前给土壤浇水以增加其黑度。
至于遮挡问题，您将需要比我最初建议更聪明的东西来处理数千种植物。也许您可以在每个阶段牺牲一些植物，测量“从头顶直接观察到的叶子面积”，然后撕下叶子，将它们分别排列在一张黑卡上，并得出总面积和可观察面积之间的经验关系。
进一步编辑
好的，所以你不能偏向对你有利的场景。您是否想过在相机上使用滤光片只允许绿光进入？或者照明使绿色物体比非绿色物体更亮？我对这个没有想法了……
最终编辑
我已经没有想法了。我认为你原来的方法，使用颜色来区分叶子和背景，是好的。由于您知道叶子的结构，因此您可以尝试模板匹配，但是您可以仅通过计算像素来估计面积和长度（或面积和长度的差异）。您可能想要研究形态学操作（例如骨架化）以从图像中导出形状度量。您可能会在有关树叶遥感（等）的文献中找到有帮助的材料。
我的印象是，您在精神上致力于实现计算机视觉系统的目标，您的目标实际上是监控植物的发育，并且您的一些想法（例如边缘检测、机器学习）无助于实现达到你正确的目标。
Since you have a great deal of control over the conditions in which you take images, bias matters in your favour. Make a mask out of black card to place around the root of the plant when you want to capture an image. Your problem reduces to one of spotting green pixels against a black background. Since you also control the position of the camera wrt the position of the plant, you should be able to arrange matters such that the pixel/mm ratio is constant across your series of images. Leaf area is then a simple matter of counting pixels.
This doesn't, yet, deal with the issue of occlusion. You could probably figure something out by taking 2 further images, elevations of the plant in 2 orthogonal planes (again using the black background) to get some idea of the shape of the plant.
EDIT after comment ...
Well, your question included the statement 'For this project I am growing a specific plant in a pot' and now you want to process thousands of the damn things. I'd still pursue the possibilities of biasing the image in your favour. For instance, if the image you posted didn't have green plastic in the background then you'd probably have a picture which could be separated, with a good degree of accuracy, by simple thresholding on the green channel of your image(s). So get rid of the green background, and water the soil before imaging to increase it's blackness.
As for the problem of occlusion, you're going to need something smarter than my original suggestion to deal with thousands of plants. Perhaps you could sacrifice a few plants at each stage, measure the 'leaf area observed from directly overhead' then rip off the leaves, arrange them separately on a piece of black card and derive an empirical relationship between total area and observable area.
FURTHER EDITING
OK, so you can't bias the scene in your favour. Have you thought of using a filter on the camera to admit only green light ? Or illumination which makes green objects brighter than not-green objects ? I'm out of ideas on this one ...
FINAL EDIT
I've run out of ideas. I think that your original approach, using colour to discriminate between leaves and background, is good. Since you know the structure of the leaves you might try template matching, but you can estimate both areas and lengths (or differences in area and length) just by counting pixels. You might want to investigate morphological operations (eg skeletonisation) for deriving shape measures from the images. You may find material in the literature on remote sensing of foliage (etc) that helps.
I get the impression that you have mentally committed to an objective of implementing a computer vision system, where your objective is really to monitor the development of the plants and that some of your ideas (eg edge-detection, machine learning) do not contribute to reaching your proper objective.
看一下均值漂移颜色分割（示例目录中的 OpenCV 包含一个示例）。您可以使用它将图像分为 2 个类别（植物和土壤），并使用它来进一步处理您的数据。
至于测量，您可能希望最初忽略遮挡效应和相机校准，而只查看图像中植物类区域的部分。
如果您想开始测量单个叶子，您可以采用“跟踪”方法，使用图像中的时间信息和空间信息。时间信息可以是先前图像中叶子的位置和大小。您可能可以应用很多技术，但如果我是您，我会从简单的开始，看看它能让您走多远。
Take a look at mean-shift color segmentation (there's an example included with OpenCV in the samples directory). You can use this to separate your image into 2 classes (plant and soil) and use that to further process your data.
As for measurement, you may want to ignore occlusion effects and camera calibration initially and just look at portion of area in image that is the plant class.
If you want to get down to measuring individual leafs, you may take a "tracking" approach where you use the temporal information as well as the spatial information in the image. The temporal information may be the location and size of the leaf in the previous image. There's probably lots of techniques you could apply, but I'd start simple if I were you and see how far it gets you.
PlantCV 指出，您对设置的控制越多，您在软件中要做的工作就越少。
如果您控制播种位置（而不是仅仅将它们散布在托盘上），您可以遮盖幼苗周围并消除背景杂乱。这对于处理单个植物以及在不同图像中查找相同植物也有很大帮助。重叠的植物会造成一种不可能的情况，无论是当你将种子和芽苗撒在彼此旁边，还是当它们超出了你的间距大小时。您必须决定种植是为了获得最佳观察效果，还是为了最佳生长。两者互不兼容。
根据我的经验，您使用颜色 (cv2.inRange) 作为主要步骤是正确的，然后继续进行 Canny 边缘检测。从那里，您可以获得轮廓。
我猜你正在使用拟南芥，所以你可能会寻找圆形图案来计算叶子（不适用于生菜）。一旦您分离了植物并具有（外部）轮廓，您就可以使用 cv2.contourArea 和 cv2.minAreaRectangle 来获取一些基本指标。
这是我正在工作的路线。
PlantCV points out that the more you control your setup, the less work you have to do in the software.
If you control your seeding position (rather than just scattering them over the tray), you can mask out around the seedlings and get rid of background clutter. This is also a significant help for working with individual plants, and finding the same plant in different images. Overlapping plants will make for an impossible situation, either when you scatter seed and the sprout next to each other, or when they exceed the size of your spacing. You have to decide if you are planting for the best observations, or optimal growth. The two are not mutually compatible.
In my experience, you are correct to use color (cv2.inRange)as the main step, then move on to Canny edge detection. From there, you can get contours.
I am guessing you are using Arabadopsis, so you might look for a circular pattern to count leaves (doesn't work with lettuce). Once you have separated your plants and have (outer) contours, you can then use cv2.contourArea and cv2.minAreaRectangle to get some basic metrics.
This is the route I am working.
这是一个活跃的研究领域。我推荐以下论文：
Scharr 等人。 2016：植物表型中的叶子分割：整理研究（pdf）
Bell和 Dee 2016：观察植物生长——立场文件关于计算机视觉和拟南芥。 http://doi.org/10.1049/iet-cvi.2016.0127
PlantCV 有一些通过距离变换和分水岭分割叶子的设施，我想添加更多。请参阅我们的预印本，其修订版本很快将在 PeerJ 上发布。
This is an active area of research. I recommend the following papers:
Scharr et al. 2016: Leaf segmentation in plant phenotyping: a collation study (pdf)
Bell and Dee 2016: Watching plants grow–a position paper on computer vision and Arabidopsis thaliana. http://doi.org/10.1049/iet-cvi.2016.0127
PlantCV has some facilities for segmenting leaves with distance transforms and watersheds, and I would like to add more. See our preprint, a revised version of which will be published in PeerJ soon.