使用 SIFT 实现增强现实
我遇到过许多 AR 库/SDK/API，它们都是基于标记的，直到我发现 这个视频，从描述和评论来看，他似乎正在使用 SIFT 来检测物体并跟踪它。
我需要为 Android 执行此操作，因此我需要用纯 Java 完整实现 SIFT。
我愿意这样做，但我首先需要知道 SIFT 如何用于增强现实。
我可以利用你提供的任何信息。
I've come across MANY AR libraries/SDKs/APIs, all of them are marker-based, until I found this video, from the description and the comments, it looks like he's using SIFT to detect the object and follow it around.
I need to do that for Android, so I'm gonna need a full implementation of SIFT in pure Java.
I'm willing to do that but I need to know how SIFT is used for augmented reality first.
I could make use of any information you give.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（6）
一如既往，维基百科是一个很好的起点：http://en.wikipedia.org/ wiki/Scale-invariant_feature_transform，但请注意 SIFT 已获得专利。
As always, Wikipedia is a good place to start from : http://en.wikipedia.org/wiki/Scale-invariant_feature_transform, but note that SIFT is patented.
在我看来，尝试为便携式设备实现 SIFT 是疯狂的。 SIFT 是一种图像特征提取算法，其中包含复杂的数学运算，当然需要大量的计算能力。 SIFT 也已获得专利。
不过，如果你确实想继续这项任务，你应该首先做一些研究。 您需要检查以下内容：
为什么选择 Android NDK？ 因为通过在 Java 应用程序使用的 C 库中实现该算法，您可能会获得更显着的性能提升。
在开始任何事情之前，请确保您进行了研究，因为如果半途才意识到图像特征提取算法对于 Android 手机来说实在是太多了，那将是一件很遗憾的事情。 实现这样一种能够提供良好结果并在可接受的时间内运行的算法本身就是一项认真的努力，更不用说使用它来创建 AR 应用程序了。
就像如何将其用于 AR 一样，我猜想在图像上运行算法所获得的描述符必须与中央数据库中保存的数据相匹配。 然后可以将结果显示给用户。 从 SURF 收集的图像的特征应该可以描述它，以便可以使用这些特征来识别它。 我对此并没有真正的经验，但网络上总是有资源。 您可能想从通用的东西开始，例如对象识别。
祝你好运:)
In my opinion, trying to implement SIFT for a portable device is madness. SIFT is an image feature extraction algorithm, which includes complex math and certainly requires a lot of computing power. SIFT is also patented.
Still, if you indeed want to go forth with this task, you should do quite some research at first. You need to check things like:
Why Android NDK? Because you'll probably have a much more significant performance gain by implementing the algorithm in a C library that's being used by your Java application.
Before starting anything, make sure you do that research cause it would be a pity to realize halfway that the image feature extraction algorithms are just too much for an Android phone. It's a serious endeavor in itself implementing such an algorithm that provides good results and runs in an acceptable amount of time, let alone using it to create an AR application.
As in how you would use that for AR, I guess that the descriptor you get from running the algorithm on an image would have to be matched against with data saved in a central database. Then the results can be displayed to the user. The features of an image gathered from SURF are supposed to describe it such as that it can be then identified using those. I'm not really experienced on doing that but there's always resources on the web. You'd probably wanna start with generic stuff such as Object Recognition.
Best of luck :)
我已经尝试过针对 330Mhz Symbian 移动设备的 SURF，即使进行了所有优化和查找表，它仍然太慢。 而且 SIFT 应该更慢。 现在每个人都在手机上使用 FAST。 无论如何，特征提取不是最大的问题。 其中通信和清除误报是比较困难的。
快速链接
http://svr-www.eng.cam.ac .uk/~er258/work/fast.html
I have tried SURF for 330Mhz Symbian mobile and it was still too slow even with all optimizations and lookup tables. And SIFT should be even more slow. Everyone using FAST for mobiles now. Anyway feature extraction is not a biggest problem. Correspondence and clearing false positive in it is more difficult.
FAST link
http://svr-www.eng.cam.ac.uk/~er258/work/fast.html
如果我在你那里，我会研究 SIFT 功能如何（以及为什么）工作（正如所说，它的维基百科页面提供了一个很好的通俗解释，有关更多详细信息，请查看科学论文（链接到维基百科） ），然后构建适合您口味的自己的变体； ie 在性能和 CPU 负载之间实现了应用程序所需的最佳平衡。
例如，我认为高斯平滑可能会被一些更快的平滑方式取代。
另外，当您构建自己的变体时，您与专利没有任何关系（已经有很多变体，例如 GLOH）。
If I where you, I'd look into how (and why) the SIFT feature works (as was said, its wikipedia-page offers a good cochise explanation, and for more details check the science paper (which is linked to at wikipedia)), and then build your own variant that suits your taste; i.e. has the optimal balance between performance and cpu-load, needed for your application.
For instance, I think Gaussian smoothing might be replaced by some faster way of smoothing.
Also, when you build your own variant, you don't have anything to do with patents (there already are lots of variants, like GLOH).
我建议您首先查看 OpenCV 库中已实现的功能，其中包括 SURF、MSER 等：
http://opencv.willowgarage.com/documentation/cpp/feature_detection.html
这对于您的应用程序来说可能足够了，并且比 SIFT 更快。 如上所述，SIFT 已获得专利。
另外，首先在移动平台上进行性能测试，只需提取每一帧的特征，这样您就可以知道哪些可以实时运行，哪些不能实时运行。
I would recommend you to start by looking at the features already implemented in the OpenCV library, which include SURF, MSER and others:
http://opencv.willowgarage.com/documentation/cpp/feature_detection.html
This might be enough for your application and are faster than SIFT. And as mentioned above, SIFT is patented.
Also, start by making performance tests in your mobile platform, just by extracting the features at every frame, this way you'll have an idea which ones can run real-time or not.
您是否在 Android 端口上尝试过 OpenCV 的 FAST 实现？ 我已经测试过它，它运行得非常快。
您还可以计算检测到的 FAST 关键点周围的简化直方图描述符。 我听说过 SIFT 的 3x3 而不是标准的 4x4。 如果您使用 NEON 指令对其进行大量优化，则很有可能实时工作。 否则，我会推荐一些快速而简单的方法，例如围绕关键点的补丁的平方和或绝对差，速度非常快。
SIFT 并不是万能药。 对于实时视频应用程序来说，这通常是多余的。
Have you tried OpenCV's FAST implementation in the Android port? I've tested it out and it runs blazingly fast.
You can also compute reduced histogram descriptors around the detected FAST keypoints. I've heard of 3x3 rather than standard 4x4 of SIFT. That has a decent chance of working in real time if you optimize it heavily with NEON instructions. Otherwise, I'd recommend something fast and simple like sum of squared or absolute differences for a patch around the keypoints which are very fast.
SIFT is not a panacea. For real time video applications, it's usually overkill.