使用Arkit中的VisionKit面对位置

发布于 2025-02-13 13:53:28 字数 2255 浏览 0 评论 0原文

我添加了visionKitarscnview上的face检测,它驾驶室检测脸,在这里我如何做我

    public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
        
        let faceDetectionRequest = VNDetectFaceRectanglesRequest(completionHandler: { (request: VNRequest, error: Error?) in
            DispatchQueue.main.async {
                self.faceLayers.forEach { drawing in
                    drawing.removeFromSuperlayer()
                }
                if let observations = request.results as? [VNFaceObservation] {
                    self.handleFaceDetectionObservations(observations: observations)
                }
            }
        })
        
        guard let capturedImage = sceneView.session.currentFrame?.capturedImage else { return }
        let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: capturedImage, orientation: .leftMirrored, options: [:])

        do {
            try imageRequestHandler.perform([faceDetectionRequest])
        } catch {
            print("perform fail, error: ", error.localizedDescription)
        }
    }


    fileprivate func handleFaceDetectionObservations(observations: [VNFaceObservation]) {
        for observation in observations {
            
            let newWidth = sceneView.bounds.width * observation.boundingBox.width
            let newHeight = sceneView.bounds.height * observation.boundingBox.height
            let newX = sceneView.bounds.width * observation.boundingBox.origin.x
            let newY = sceneView.bounds.height * observation.boundingBox.origin.y
            let faceRectConverted = CGRect(x: newX, y: newY, width: newWidth, height: newHeight)
            
            let faceRectanglePath = CGPath(rect: faceRectConverted, transform: nil)
            
            let faceLayer = CAShapeLayer()
            faceLayer.path = faceRectanglePath
            faceLayer.fillColor = UIColor.black.cgColor
            self.faceLayers.append(faceLayer)
            self.sceneView.layer.addSublayer(faceLayer)
            
        }
    }

在这里唯一的问题是视图中的面部位置,它是计算错误。看来问题来自相机镜像,当右转时,面部矩形向左或脸部抬高时,矩形向下倾斜。我不知道如何进行正确的计算以将观察<代码> rect 与sceneView的正确位置联系起来。谁能帮我! 在景观上也有同样的问题,矩形高度在那里更紧凑... 谢谢

I added visionKit face detection on an ARSCNView, it cab detect the face, here how I did that

    public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
        
        let faceDetectionRequest = VNDetectFaceRectanglesRequest(completionHandler: { (request: VNRequest, error: Error?) in
            DispatchQueue.main.async {
                self.faceLayers.forEach { drawing in
                    drawing.removeFromSuperlayer()
                }
                if let observations = request.results as? [VNFaceObservation] {
                    self.handleFaceDetectionObservations(observations: observations)
                }
            }
        })
        
        guard let capturedImage = sceneView.session.currentFrame?.capturedImage else { return }
        let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: capturedImage, orientation: .leftMirrored, options: [:])

        do {
            try imageRequestHandler.perform([faceDetectionRequest])
        } catch {
            print("perform fail, error: ", error.localizedDescription)
        }
    }


    fileprivate func handleFaceDetectionObservations(observations: [VNFaceObservation]) {
        for observation in observations {
            
            let newWidth = sceneView.bounds.width * observation.boundingBox.width
            let newHeight = sceneView.bounds.height * observation.boundingBox.height
            let newX = sceneView.bounds.width * observation.boundingBox.origin.x
            let newY = sceneView.bounds.height * observation.boundingBox.origin.y
            let faceRectConverted = CGRect(x: newX, y: newY, width: newWidth, height: newHeight)
            
            let faceRectanglePath = CGPath(rect: faceRectConverted, transform: nil)
            
            let faceLayer = CAShapeLayer()
            faceLayer.path = faceRectanglePath
            faceLayer.fillColor = UIColor.black.cgColor
            self.faceLayers.append(faceLayer)
            self.sceneView.layer.addSublayer(faceLayer)
            
        }
    }

The only issue that I have here is the face position in the view, it's calculated wrong. Looks like the problem come from camera mirroring, when the goes right, the face rectangular goes left, or when the face goes up, the rectangular goes down. I don't know how to do the right calculation to tie observation rect to the right place in sceneView . Could anyone help me on that!
Also have a same problem on landscape, the rectangular height is more compact there...
Thanks

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文