我有一个想法，我需要能够从渲染的三维数字电影中识别某些对象或模型。
经过有限的研究，我现在知道我需要的是计算机视觉领域中的特征检测。
所以，我想做的是：
- 创建电影中某个角色的几个屏幕截图（例如前/后/左/右）
- 播放电影时播放电影
- 不断创建电影的新屏幕截图
- ，为每个屏幕截图 ，执行特征检测（SIFT？，使用 openCV？）以查看是否存在任何我们的角色外观（如果角色距离较远并因此显得较小，或者角色例如躺着，则仍然必须识别它们）。
- 每当找到该角色时发出通知
这可以通过 OpenCV 实现，对吗？
“问题”是我必须学习 c++ 或 python 来开发这个应用程序。如果我的电影和屏幕截图适用于我想做的事情，这不是问题。
所以，我想先测试一下我的电影截图。是否有 OpenCV 的 GUI 版本，我可以输入测试数据，然后手动执行其特征检测算法作为原型设计的手段？
如有任何反馈，我们将不胜感激。谢谢。
I had an idea for which I need to be able to recognize certain objects or models from a rendered three dimensional digital movie.
After limited research, I know now that what I need is called feature detection in the field of Computer Vision.
So, what I want to do is:
- create a few screenshots of a certain character in the movie (eg. front/back/leftSide/rightSide)
- play the movie
- while playing the movie, continuously create new screenshots of the movie
- for each screenshot, perform feature detection (SIFT?, with openCV?) to see if any of our character appearances are there (they must still be recognized if the character is further away and thus appears smaller, or if the character is eg. lying down).
- give a notice whenever the character is found
This would be possible with OpenCV, right?
The "issue" is that I would have to learn c++ or python to develop this application. This is not a problem if my movie and screenshots are applicable for what I want to do.
So, I would like to first test my screenshots of the movie. Is there a GUI version of OpenCV that I can input my test data and then execute it's feature detection algorithms manually as a means of prototyping?
Any feedback is appreciated. Thanks.
发布评论
评论（5）
OpenCV 没有 GUI 可以做你想做的事。您将能够使用 OpenCV 来解决您问题的某些方面，但没有现成的解决方案在等着您。
虽然绝对有可能解决您的问题，但这个问题的学习曲线相当长。如果您是专业人士，那么除了自己学习之外，另一种选择是聘请专家为您做这件事。这会花费金钱，但可以节省您的时间。
编辑
就模板匹配而言，您通常不会使用它来解决这样的问题，因为您正在寻找的是改变外观和形状。实际上没有任何“要设置的动态参数”。您可以尝试的最接近的方法是拥有一个大量模板集合，尝试涵盖您的目标可能采用的预期形式。但这并不是一个优雅的解决方案。而且它无法扩展。
接下来，谈谈关于人脸识别的问题。这是相关的，但大多数面部识别应用程序都处理受控环境：照明、距离、姿势、角度等。在受控环境之外，面部检测效率会显着下降。如果您正在检测电影中的物体，那么您的环境并没有真正受到控制。
您可能想首先尝试一个更简单的问题，即准确检测角色的位置，而不是确定他们是谁（本质上是视频监控）。虽然听起来很简单，但您会发现它对于任意场景来说实际上并不简单。解决该问题的结果可能有助于识别字符。
There is no GUI of OpenCV able to do what you want. You will be able to use OpenCV for some aspects of your problem, but there is no ready-made solution waiting there for you.
While it's definitely possible to solve your problem, the learning curve for this problem is quite long. If you're a professional, then an alternative to learning about it yourself would be to hire an expert to do it for you. It would cost money, but save you time.
EDIT
As far as template matching goes, you wouldn't normally use it to solve such a problem because the thing you're looking for is changing appearance and shape. There aren't really any "dynamic parameters to set". The closest thing you could try is have a massive template collection that would try to cover the expected forms that your target may take. But it would hardly be an elegant solution. Plus it wouldn't scale.
Next, to your point about face recognition. This is kind of related, but most facial recognition applications deal with a controlled environment: lighting, distance, pose, angle, etc. Outside of that controlled environment face detection effectiveness drops significantly. If you're detecting objects in a movie, then your environment isn't really controlled.
You may want to first try a simpler problem of accurately detecting where the characters are, without determining who they are (video surveillance, essentially). While it may sound simple, you'll find that it's actually non-trivial for arbitrary scenes. The result of solving that problem may be useful in identifying the characters.
Find-Object .usherbrooke.ca/mediawiki-introlab/index.php/Mathieu_Labb%C3%A9" rel="nofollow">Mathieu Labbé。这对我开始了解描述符非常有帮助，因为您可以在视频运行时更改它们以查看会发生什么。
这可能为时已晚，但可能会帮助其他人寻找解决方案。
There is Find-Object by Mathieu Labbé. It was very helpful for me to start getting an understanding of the descriptors since you can change them while your video is running to see what happens.
This is probably too late, but might help someone else looking for a solution.
好吧，使用 OpenCV，您可以获取视频文件的一帧并对其进行任何计算。
您可以采用多种不同的方法来检测该图像上的角色，但要让它灵活起来并不容易，因此您甚至可以在该人躺在地板上时检测到该人，例如，如果您只输入该角色站立的参考图像。
基本上，您可以尝试从参考图片集中提取所有重要特征，并使用（在您的情况下有监督的）学习算法来获取该字符的良好特征向量以进行分类。
然后，您需要编写播放视频的代码，并每隔 500 毫秒（或您想要的其他时间）获取一个视频帧，获取您认为是该角色的对象的分段，并将其与您获得的参考值进行比较从你的学习算法中。如果存在匹配，您的代码可以大喊“Yehaaawww！”或者做其他事情...
但这一切都取决于您希望它有多灵活。您还可以尝试模板匹配或互相关，这基本上会在框架上移动参考图像并检查两个部分的相等程度。但不幸的是，它对旋转、变形或其他噪音非常敏感……所以如果那个人躺着，你就抓不到那个人。我怀疑你能否实时完成所有这些计算......
基本上：是的 OpenCV 非常适合用于你的图像处理/计算机视觉任务。但它提供了很多方法和途径，您需要找到一种适合您的图像的方法...但这不是一个微不足道的任务...
希望有所帮助...
Well, using OpenCV you would of taking a frame of a video file and do any computations on it.
You can do several different methods of detecting a character on that image, but it's not so easy to have it as flexible so you can even get that person if it's lying on the floor for example, if you only entered reference images of that character standing.
Basically you could try extracting all important features from your set of reference pictures and have a (in your case supervised) learning algorithm that gets a good feature-vector of that character for classification.
You then need to write your code that plays the video and which takes a video frame let's say each 500ms (or other as you desire), gets a segmentation of the object you thing would be that character and compare it with the reference values you get from your learning algorithm. If there's a match, your code can yell "Yehaaawww!" or do other things...
But all this depends on how flexible you want this to be. You could also try a template match or cross-correlation which basically shifts the reference image(s) over the frame and checks how equal both parts are. But this unfortunately is very sensitive for rotation, deformations or other noise... so you wouldn't get that person if its i.e. laying down. And I doubt you can get all those calculations done in realtime...
Basically: Yes OpenCV is good to use for your image processing/computer vision tasks. But it offers a lot of methods and ways and you'd need to find a way that works for your images... it's not a trivial task though...
Hope that helps...
您是否尝试过查看牛津视觉几何小组的一些工作？
他们的Video Google系统在很大程度上描述了您想要的内容，实例检测。
他们在为电视节目中的人物命名方面的工作也是非常相关。包含可从 Matlab 运行的面部检测和面部特征管道。你熟悉Matlab吗？
Have you tried looking at some of the work of the Oxford visual geometry group?
Their Video Google system describes to a large extent what you want, instance detection.
Their work into Naming People in TV shows is also pretty relevant. A face detection and facial feature pipeline is included that can be run from Matlab. Are you familiar with Matlab?
您是否尝试过像 Cassandra 这样的计算机视觉框架？在那里，您只需点击几下鼠标就可以做到这一点。
Have you tried computer vision frameworks like Cassandra? There you can exactly do that just by some mouse clicks.