使用 Kinect 去除背景:体形周围的噪声抑制

发布于 2024-12-03 12:16:20 字数 766 浏览 1 评论 0原文

目的是在不同的背景上显示人物(也称为背景去除)。

我使用 Kinect 和 Microsoft 的 Beta Kinect SDK 来执行此操作。在深度的帮助下,背景被过滤,我们只得到人物的图像。

这做起来非常简单,我们可以在互联网上随处找到执行此操作的代码。然而,深度信​​号是有噪声的,我们得到的像素不属于所显示的人。

我应用了边缘检测器来查看它是否有用,目前我得到了这个:

这是另一个没有边缘检测的:

在此处输入图像描述

我的问题是:我可以通过哪种方式消除人周围这些嘈杂的白色像素?

我尝试了形态学操作,但身体的某些部分被擦除,但仍然留下白色像素。

该算法不需要是实时的,我可以在按下“保存图像”按钮时应用它。

编辑1:

我只是尝试用形状边框上最接近的框架进行背景减除。你看到的单个像素在闪烁,这意味着它是噪音,我可以轻松消除它们。

编辑 2:

项目现已结束,这就是我们所做的:使用 OpenNI 驱动程序手动校准 Kinect,它直接提供红外图像。结果确实不错,但每次校准都是针对每个 Kinect 的。

然后,我们在边框上应用了一点透明度,结果看起来非常漂亮!不过我无法提供图片。

The objective is to display the person on a different background (aka background removal).

I'm using the Kinect with Microsoft's Beta Kinect SDK to do so. With help of the depth, the background is filtered and we get only the image of the person.

This is pretty simple to do, and we can find the code that does that everywhere on the Internet. However, the depth signal is noisy, and we get pixels which do not belong to the person that are displayed.

I applied an edge detector to see if it was useful, and I currently get this:

Here's another without edge detection:

enter image description here

My question is: Which way can I get rid of these noisy white pixels around the person?

I tried morphological operations, but some parts of the body are erased and still leave white pixels behind.

The algorithm doesn't need to be real-time, I can just apply it when I press a 'Save image' button.

Edit 1:

I just tried to do background substraction with the closest frames on the shape border. The single pixels you see are flickering, which means it is noise and I can get easily get rid of them.

Edit 2:

The project is now over, and here's what we did: manual calibration of the Kinect by using the OpenNI driver, which provides directly the infrared image. The result is really good, but each calibration is specific to each Kinect.

Then, we applied a little transparency on the borders, and the result looks really nice! I can't provide pictures, however.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

早乙女 2024-12-10 12:16:20

您的问题不仅仅是嘈杂的白色像素。你也错过了这个人的重要部分,例如他右手的一部分。我建议对深度数据的阈值更加保守(允许更多误报)。这会给你带来更多嘈杂的像素,但至少你会得到完整的人。

为了消除噪声像素,我可以考虑以下几点:

  • 羽化外部像素(如果使用 Alpha 通道,则降低它们的强度/增加它们的透明度)
  • 平滑图像,对平滑后的图像执行边缘检测图像,然后将这些边缘与原始清晰图像一起使用。
  • 进行一些皮肤区域检测来标记肯定属于一个人的部分。请参阅 YUV 色彩空间中的皮肤检测?肤色检测
  • 对于衣服,使用色调和饱和度图像。如果您知道 T 恤的颜色(或者至少知道它不是中性色),那么它会很容易脱颖而出。如果您不知道这些信息,那么可能值得使用其他帧建立人物模型(如果您的视频中有一个大的灰色斑点在移动,则您的拍摄对象很可能穿着一件灰色衬衫)

这些方法并不相互排斥,因此可能值得尝试将它们组合起来。如果我想到什么,我会在这里发帖。

Your problem isn't just the noisy white pixels. You're missing significant parts of the person as well, e.g. part of his right hand. I'd recommend being more conservative with your thresholding of the depth data (allow more false positives). This would give you more noisy pixels, but at least you'd have the person in their entirety.

To get rid of the noisy pixels, I can think of a couple of things:

  • Feather the outer pixels (reduce them in intensity/increase their transparency if you're using an alpha channel)
  • Smooth the image, perform the edge detection on the smoothed image, then use these edges with your original sharp image.
  • Do some skin region detection to mark parts that definitely belong to a person. See skin detection in the YUV color space? and Skin Color Detection
  • For clothes, work with the hue and saturation image. If you know the color of the t-shirt (or that at least that it's not a neutral color), then this will stand out easily. If you don't know this information, then it may be worth building up a model of the person using the other frames (if there's a big gray blob that's moving around in your video, chances are that your subject is wearing a gray shirt)

The approaches aren't mutually exclusive so it may be worth trying to do them in combination. If I think of anything else, I'll post back here.

蘸点软妹酱 2024-12-10 12:16:20

如果没有其他方法来解决边缘抖动,您可以随时尝试抗锯齿作为后处理。

If there is no other way of resolving the jitter on the edges you could always try anti-alias as post-process.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文