OpenCV 进程运行时 QMainWindow 没有响应

Question

我在 QtGui 应用程序中使用自定义 OpenCV VideoProcessor-Class。我的主窗口有 2 个 ViewerWidget，用于显示 VideoProcessor 对象生成的输入和输出帧。 VideoProcessor-Object 获取这些 ViewerWidget 上的指针，以便在这些 Widget 上显示处理后的帧。

当我启动应用程序时，GUI 窗口中的所有内容都会响应用户输入。但是当我开始处理时它停止响应。我什至无法关闭窗口或从应用程序菜单中选择某些内容。处理显示正确的输出并继续运行，但窗口不再响应。

这是开始处理的主窗口插槽：

void MainWindow::on_actionStart_Capture_triggered()
{
    // Create instance
    p = new VideoProcessor();
          // Open video file
          p->setInput(0);
          // Declare a window to display the video
          p->displayInput("Current Frame");
          p->displayOutput("Output Frame");
          // Play the video at the original frame rate
          p->setDelay(1000./p->getFrameRate());
          // Set the frame processor callback function
          p->setFrameProcessor(canny);
          // Start the process
          p->run(cvWidgetIn, cvWidgetOut);
}

这是视频处理器。该文件来自 OpenCV Cookbook，我将其更改为在下面代码末尾的 run() 函数中获取指向我的 ViewerWidgets 的指针。

#if !defined VPROCESSOR
#define VPROCESSOR

#include <iostream>
#include <iomanip>
#include <sstream>
#include <string>
#include <vector>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include "cvwidget.h"

// The frame processor interface
class FrameProcessor {

  public:
    // processing method
    virtual void process(cv:: Mat &input, cv:: Mat &output)= 0;
};

class VideoProcessor {

  private:

      // the OpenCV video capture object
      cv::VideoCapture capture;
      // the callback function to be called 
      // for the processing of each frame
      void (*process)(cv::Mat&, cv::Mat&);
      // the pointer to the class implementing 
      // the FrameProcessor interface
      FrameProcessor *frameProcessor;
      // a bool to determine if the 
      // process callback will be called
      bool callIt;
      // Input display window name
      std::string windowNameInput;
      // Output display window name
      std::string windowNameOutput;
      // delay between each frame processing
      int delay;
      // number of processed frames 
      long fnumber;
      // stop at this frame number
      long frameToStop;
      // to stop the processing
      bool stop;

      // vector of image filename to be used as input
      std::vector<std::string> images; 
      // image vector iterator
      std::vector<std::string>::const_iterator itImg;

      // the OpenCV video writer object
      cv::VideoWriter writer;
      // output filename
      std::string outputFile;

      // current index for output images
      int currentIndex;
      // number of digits in output image filename
      int digits;
      // extension of output images
      std::string extension;

      // to get the next frame 
      // could be: video file; camera; vector of images
      bool readNextFrame(cv::Mat& frame) {

          if (images.size()==0)
              return capture.read(frame);
          else {

              if (itImg != images.end()) {

                  frame= cv::imread(*itImg);
                  itImg++;
                  return frame.data != 0;
              }
          }
      }

      // to write the output frame 
      // could be: video file or images
      void writeNextFrame(cv::Mat& frame) {

          if (extension.length()) { // then we write images

              std::stringstream ss;
              ss << outputFile << std::setfill('0') << std::setw(digits) << currentIndex++ << extension;
              cv::imwrite(ss.str(),frame);

          } else { // then write video file

              writer.write(frame);
          }
      }

  public:

      // Constructor setting the default values
      VideoProcessor() : callIt(false), delay(-1), 
          fnumber(0), stop(false), digits(0), frameToStop(-1), 
          process(0), frameProcessor(0) {}

      // set the name of the video file
      bool setInput(std::string filename) {

        fnumber= 0;
        // In case a resource was already 
        // associated with the VideoCapture instance
        capture.release();
        images.clear();

        // Open the video file
        return capture.open(filename);
      }

      // set the camera ID
      bool setInput(int id) {

        fnumber= 0;
        // In case a resource was already 
        // associated with the VideoCapture instance
        capture.release();
        images.clear();

        // Open the video file
        return capture.open(id);
      }

      // set the vector of input images
      bool setInput(const std::vector<std::string>& imgs) {

        fnumber= 0;
        // In case a resource was already 
        // associated with the VideoCapture instance
        capture.release();

        // the input will be this vector of images
        images= imgs;
        itImg= images.begin();

        return true;
      }

      // set the output video file
      // by default the same parameters than input video will be used
      bool setOutput(const std::string &filename, int codec=0, double framerate=0.0, bool isColor=true) {

          outputFile= filename;
          extension.clear();

          if (framerate==0.0) 
              framerate= getFrameRate(); // same as input

          char c[4];
          // use same codec as input
          if (codec==0) { 
              codec= getCodec(c);
          }

          // Open output video
          return writer.open(outputFile, // filename
              codec, // codec to be used 
              framerate,      // frame rate of the video
              getFrameSize(), // frame size
              isColor);       // color video?
      }

      // set the output as a series of image files
      // extension must be ".jpg", ".bmp" ...
      bool setOutput(const std::string &filename, // filename prefix
          const std::string &ext, // image file extension 
          int numberOfDigits=3,   // number of digits
          int startIndex=0) {     // start index

          // number of digits must be positive
          if (numberOfDigits<0)
              return false;

          // filenames and their common extension
          outputFile= filename;
          extension= ext;

          // number of digits in the file numbering scheme
          digits= numberOfDigits;
          // start numbering at this index
          currentIndex= startIndex;

          return true;
      }

      // set the callback function that will be called for each frame
      void setFrameProcessor(void (*frameProcessingCallback)(cv::Mat&, cv::Mat&)) {

          // invalidate frame processor class instance
          frameProcessor= 0;
          // this is the frame processor function that will be called
          process= frameProcessingCallback;
          callProcess();
      }

      // set the instance of the class that implements the FrameProcessor interface
      void setFrameProcessor(FrameProcessor* frameProcessorPtr) {

          // invalidate callback function
          process= 0;
          // this is the frame processor instance that will be called
          frameProcessor= frameProcessorPtr;
          callProcess();
      }

      // stop streaming at this frame number
      void stopAtFrameNo(long frame) {

          frameToStop= frame;
      }

      // process callback to be called
      void callProcess() {

          callIt= true;
      }

      // do not call process callback
      void dontCallProcess() {

          callIt= false;
      }

      // to display the processed frames
      void displayInput(std::string wn) {

          windowNameInput= wn;
          //cv::namedWindow(windowNameInput);
      }

      // to display the processed frames
      void displayOutput(std::string wn) {

          windowNameOutput= wn;
          //cv::namedWindow(windowNameOutput);
      }

      // do not display the processed frames
      void dontDisplay() {

          cv::destroyWindow(windowNameInput);
          cv::destroyWindow(windowNameOutput);
          windowNameInput.clear();
          windowNameOutput.clear();
      }

      // set a delay between each frame
      // 0 means wait at each frame
      // negative means no delay
      void setDelay(int d) {

          delay= d;
      }

      // a count is kept of the processed frames
      long getNumberOfProcessedFrames() {

          return fnumber;
      }

      // return the size of the video frame
      cv::Size getFrameSize() {

        if (images.size()==0) {

            // get size of from the capture device
            int w= static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
            int h= static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));

            return cv::Size(w,h);

        } else { // if input is vector of images

            cv::Mat tmp= cv::imread(images[0]);
            if (!tmp.data) return cv::Size(0,0);
            else return tmp.size();
        }
      }

      // return the frame number of the next frame
      long getFrameNumber() {

        if (images.size()==0) {

            // get info of from the capture device
            long f= static_cast<long>(capture.get(CV_CAP_PROP_POS_FRAMES));
            return f; 

        } else { // if input is vector of images

            return static_cast<long>(itImg-images.begin());
        }
      }

      // return the position in ms
      double getPositionMS() {

          // undefined for vector of images
          if (images.size()!=0) return 0.0;

          double t= capture.get(CV_CAP_PROP_POS_MSEC);
          return t; 
      }

      // return the frame rate
      double getFrameRate() {

          // undefined for vector of images
          if (images.size()!=0) return 0;

          double r= capture.get(CV_CAP_PROP_FPS);
          return r; 
      }

      // return the number of frames in video
      long getTotalFrameCount() {

          // for vector of images
          if (images.size()!=0) return images.size();

          long t= capture.get(CV_CAP_PROP_FRAME_COUNT);
          return t; 
      }

      // get the codec of input video
      int getCodec(char codec[4]) {

          // undefined for vector of images
          if (images.size()!=0) return -1;

          union {
              int value;
              char code[4]; } returned;

          returned.value= static_cast<int>(capture.get(CV_CAP_PROP_FOURCC));

          codec[0]= returned.code[0];
          codec[1]= returned.code[1];
          codec[2]= returned.code[2];
          codec[3]= returned.code[3];

          return returned.value;
      }

      // go to this frame number
      bool setFrameNumber(long pos) {

          // for vector of images
          if (images.size()!=0) {

              // move to position in vector
              itImg= images.begin() + pos;
              // is it a valid position?
              if (pos < images.size())
                  return true;
              else
                  return false;

          } else { // if input is a capture device

            return capture.set(CV_CAP_PROP_POS_FRAMES, pos);
          }
      }

      // go to this position
      bool setPositionMS(double pos) {

          // not defined in vector of images
          if (images.size()!=0) 
              return false;
          else 
              return capture.set(CV_CAP_PROP_POS_MSEC, pos);
      }

      // go to this position expressed in fraction of total film length
      bool setRelativePosition(double pos) {

          // for vector of images
          if (images.size()!=0) {

              // move to position in vector
              long posI= static_cast<long>(pos*images.size()+0.5);
              itImg= images.begin() + posI;
              // is it a valid position?
              if (posI < images.size())
                  return true;
              else
                  return false;

          } else { // if input is a capture device

              return capture.set(CV_CAP_PROP_POS_AVI_RATIO, pos);
          }
      }

      // Stop the processing
      void stopIt() {

          stop= true;
      }

      // Is the process stopped?
      bool isStopped() {

          return stop;
      }

      // Is a capture device opened?
      bool isOpened() {

          return capture.isOpened() || !images.empty();
      }

      // to grab (and process) the frames of the sequence
      void run(CVWidget *inputWidget, CVWidget *outputWidget) {

          // current frame
          cv::Mat frame;
          // output frame
          cv::Mat output;

          // if no capture device has been set
          if (!isOpened())
              return;

          stop= false;

          while (!isStopped()) {

              // read next frame if any
              if (!readNextFrame(frame))
                  break;

              // display input frame
              if (windowNameInput.length()!=0) 
                 // cv::imshow(windowNameInput,frame);
              inputWidget->sendImage(&frame);


              // calling the process function or method
              if (callIt) {

                // process the frame
                if (process)
                    process(frame, output);
                else if (frameProcessor) 
                    frameProcessor->process(frame,output);
                // increment frame number
                fnumber++;

              } else {

                output= frame;
              }

              // write output sequence
              if (outputFile.length()!=0)
                  writeNextFrame(output);

              // display output frame
              if (windowNameOutput.length()!=0) 
                  //cv::imshow(windowNameOutput,output);
                  outputWidget->sendImage(&output);

              // introduce a delay
              if (delay>=0 && cv::waitKey(delay)>=0)
                stopIt();

              // check if we should stop
              if (frameToStop>=0 && getFrameNumber()==frameToStop)
                  stopIt();
          }
      }
};

#endif

Answer 1

虽然我无法找出问题的具体原因是什么，但我终于找到了一个适合我的解决方案：

正如您所建议的，我摆脱了这个 VideoProcessor-Utility 类并实现了用于处理帧序列的处理循环在主窗口内使用 QTimer 在每一帧之间进行延迟。

我制作了“ProcessorWidget”，而不是 VideoProcessor-Class。这个 GUI 小部件提供了用于参数化我迄今为止实现的每个处理器功能的所有输入控件。

所有 OpenCV 代码现在都在 ProcessorWidget 类中，它有一个公共槽 cv::Mat process(cv::Mat input)，它只接受输入帧，然后解析来自 GUI 的所有用户输入参数并处理在内部选择正确的处理器和参数。

MainWindow 现在构建 ViewerWidgets 和 ProcessorWidget 并拥有 Control 和 Timing。

处理和显示帧现在只是：

cvWidgetOut->sendImage(&processor->processFrame(input));

如果我想添加更多 OpenCV 功能，我不必更改我的主窗口或 GUI。这一切都在 ProcessorWidget 类中完成。

Answer 2

openCV 的 highgui 窗口处理它自己的事件循环 - 你不能（轻松）将它与 QMainWindows 事件循环混合。

最简单的方法是在openCV中抓取图像并使用Qimage和QPainter在Qt中显示 - QImage的24位RGB888格式与opencv的CV_8UC3兼容