ML KIT条形码扫描未检测到监视器屏幕照片中的QR码

发布于 2025-01-18 03:03:37 字数 1438 浏览 3 评论 0 原文

我正在使用 com.google.mlkit.mlkit.mlkit:barcode-scanning:barcode-scanning:bar occanning:barcode: 17.0.2 检测图片中的QR码。

从画廊获得URI后,我创建 InputImage ,然后使用 barcodescanner 来查找QR代码。当我在纸质代码上选择QR码的照片时。但是,当我在监视器屏幕代码上拍摄QR码的照片时。我该怎么办才能在监视器屏幕的照片中检测QR码?

(当我使用 camerax 进行实时QR代码检测的同一扫描仪时,它会在监视器屏幕上找到代码)

val image = InputImage.fromFilePath(context, uri)

val scanOptions =
    BarcodeScannerOptions.Builder()
        .setBarcodeFormats(
            Barcode.FORMAT_QR_CODE,
        )
        .build()

val scanner = BarcodeScanning.getClient(scanOptions)

scanner.process(image)
    .addOnSuccessListener {
        val code = it.getOrNull(0)?.rawValue
        if (code == null) {
            // code NOT found
        } else {
            // code was found
        }
    }

QR代码的示例在纸上找到

在监视器屏幕上的QR码的示例

”在此处输入图像说明”

I am using com.google.mlkit:barcode-scanning:17.0.2 to detect QR codes in the pictures.

After getting URI from the gallery I create InputImage and then process this image with BarcodeScanner to find QR codes. When I select a photo of QR codes on paper code is found. But when I take a photo of the QR code on the monitor screen code is never found. What I should do to be able to detect a QR code in a photo of a monitor screen?

(When I use the same scanner with CameraX to do live QR code detection it finds code on the monitor screen)

val image = InputImage.fromFilePath(context, uri)

val scanOptions =
    BarcodeScannerOptions.Builder()
        .setBarcodeFormats(
            Barcode.FORMAT_QR_CODE,
        )
        .build()

val scanner = BarcodeScanning.getClient(scanOptions)

scanner.process(image)
    .addOnSuccessListener {
        val code = it.getOrNull(0)?.rawValue
        if (code == null) {
            // code NOT found
        } else {
            // code was found
        }
    }

Example of QR code on paper which is found

enter image description here

Example of QR code on the monitor screen which is NOT found

enter image description here

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

风情万种。 2025-01-25 03:03:37

可能是您正在与moiréformation。根据QR检测算法,Moiré效应引入的高频可以使检测器脱离轨道。令人沮丧的是,通常是更好的 qrcode检测器被莫伊尔模式击败。

一个良好的解决方法是:

  • 以最高分辨率拍摄图片,您可以
  • 执行图片
  • 增加的模糊与最大形成对比,如果可能
  • ((选择))运行Sigma阈值,或者只是将所有像素以低于32至0的Luma组件重写所有像素所有以上224至255的人。

进行大约相同操作的另一种方法是以

  • 最高的分辨率拍摄图片,您可以
  • 增加与最大对比的对比,如果可能的话,
  • 将图片删除到一个分辨率,该分辨率降低了

第二种方法,第二种方法给出了较差的结果,但通常可以用设备原始词实现。

监视器问题的另一个来源(据我所知,在您的照片中都没有)是刷新率。有时,您会发现QR码实际上是图片上半部分中的一个过度曝光的QRCODE,并且在图片的下半部分是一个未充满刺激的QRCODE。两者都没有得到认可。这种效果是由于监视器的刷新率和策略造成的,并且不容易解决 - 您可以尝试降低监视器的光度以增加曝光时间,直到超过1/50或1/25秒的时间,或从更远的地方拍摄图片离开并使用数字缩放。现代监视器的刷新率更高,实际上在其居住时间内刷新的时间更高,因此不应该发生这种情况。但是,使用旧的模拟显示器,但每次都会发生

第三个疯狂的方式

是偶然发现的,但是即使在QR SDK或图书馆提供一些小的额外装饰的情况下,它也可以很好地奏效。

  1. 以最高帧速率(25 fps?),以大约1秒长的视频以约1秒的长度。
  2. 从中间(例如13号)帧中提取三个QR“ Waypoints” - 您的SDK中可能有一个低级函数,称为“ containsqrcode()”可以做到这一点。如果它返回 true ,则找到路点并返回其坐标以允许执行缩放/估计值。它可能会返回一个置信度(“此图片似乎包含概率x%的QR码”)。这些是应用程序在候选QR码周围显示框架或红点的API。如果您的SDK没有这些API,对不起...您不幸。
  3. 立即在(第12和14位),然后在第11和15位,然后再获取框架,依此类推。如果这些中的任何一个都返回有效的QR码,则您是免费的。
  4. 如果在足够的框架中找到了QR码(即使未正确解码),但是Waypoint坐标差异很大,则手不稳定 - 对用户说。
  5. 如果您有足够的框架,坐标差异很小,您可以将其居中并对齐,而平均框架。然后在结果图像上运行实际的QRCODE识别。这消除了100%的Moiré效果,并且也大大降低了监视器的停留噪声,几乎没有信息丢失。结果比分辨率更改要好得多,分辨率更改并不容易在分辨率更改时重置相机的设备(某些)设备。

这在嘈杂,振动富的环境中运行的19美元ESP32 IoT设备(它从移动传输色带上的纸箱盒子的相机图像中获取了QR码)。

Chances are that you're fighting against Moiré effect. Depending on the QR detection algorithm, the high frequencies introduced by the Moiré effect can throw the detector off its track. Frustratingly, it is often the better QRcode detectors that are defeated by Moiré patterns.

A good workaround is:

  • take the picture at the highest resolution you can
  • perform a blurring of the picture
  • increase contrast to the max, if possible
  • (optionally) run a sigma thresholding, or just rewrite all pixels with a luma component below 32 to 0, all those above 224 to 255.

Another way of doing approximately the same operation is

  • take the picture at the highest resolution you can
  • increase contrast to the max, if possible
  • downsample the picture to a resolution which is way lower

The second method gives worse results, but usually can be implemented with device primitives.

Another source of problems with monitors (not in your picture as far as I can see) is the refresh rate. Sometimes, you'll find that the QR code is actually an overexposed QRcode in the upper half of the picture and an underexposed QRcode in the bottom half of the picture. Neither are recognized. This effect is due to the monitor's refresh rate and strategy and is not easy to solve - you can try lowering the monitor's luminosity to increase exposure time, until it exceeds 1/50th or 1/25th of a second, or take the picture from farther away and use digital zooming. Modern monitors have higher refresh rates and actually refresh at more than their own dwell time, so this should not happen; with old analog monitors however it will happen every time.

A third, crazy way

This was discovered half by chance, but it works really well even on cheap hardware provided the QR SDK or library supplies some small extra frills.

  1. Take a video of about 1 second length at the highest frame rate you can get (25 fps?).
  2. From the middle (e.g. 13th) frame, extract the three QR "waypoints" - there might be a low-level function in your SDK called "containsQRCode()" that does this. If it returns true, the waypoints were found and their coordinates are returned to allow performing scaling/estimates. It might return a confidence figure ("this picture seems to contain a QR code with probability X%"). These are the APIs used by apps to show a frame or red dots around candidate QR codes. If your SDK doesn't have these APIs, sorry... you're out of luck.
  3. Get the frames immediately before and after (12th and 14th), then the 11th and 15th, and so on. If any of these returns a valid QR code, you're home free.
  4. If the QR code is found (even if not correctly decoded) in enough frames, but the waypoint coordinates vary much, the hand is not steady - say so to the user.
  5. If you have enough frames with coordinates that vary little, you can center and align on those, and average the frames. Then run the real QRCode recognition on the resulting image. This gets rid of 100% of the Moiré effect, and also drastically reduces monitor dwell noise with next to no information loss. The results are way better than the resolution change, which isn't easy to perform on (some) devices that reset the camera upon resolution change.

This worked on a $19 ESP32 IoT device operating in a noisy, vibration-rich environment (it acquires QR codes from a camera image of carton boxes on a moving transport ribbon).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文