Segmentation Fault while using MPI and OpenCV toge

问题:

I am trying to learn MPI in C++. I have some knowledge of OpenCV so I tried writing a program using both MPI and OpenCV. This may sound stupid but for the purpose of learning I tried capturing an image from webcam on thread 0 and passed the image to thread 1 for converting to grayscale and displaying the grayscale image.

This is how I compile the code:
mpic++ opencv.cpp `pkg-config opencv --libs`

The code compiles sucessfully but when I run the executable, an image shows up on the screen for a fraction of a second and this is what I see on the terminal

~/mpi$ mpirun -np 2 ./a.out
libv4l2: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl S_FMT
libv4l2: error setting pixformat: Device or resource busy
libv4l1: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl VIDIOCSPICT

[arch:09670] *** Process received signal *** 
[arch:09670] Signal: Segmentation fault (11)
[arch:09670] Signal code: Address not mapped (1) 
[arch:09670] Failing at address: 0x218ac50
[arch:09670] [ 0] /usr/lib/libpthread.so.0(+0x10740)[0x7f422fcac740]
[arch:09670] [ 1] /usr/lib/libopencv_core.so.2.4(_ZNK2cv11_InputArray6getMatEi+0x203)[0x7f4233c8c113]
[arch:09670] [ 2] /usr/lib/libopencv_imgproc.so.2.4(_ZN2cv8cvtColorERKNS_11_InputArrayERKNS_12_OutputArrayEii+0x50)[0x7f4232c25de0]
[arch:09670] [ 3] ./a.out[0x408f54]
[arch:09670] [ 4] /usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f422e9e9800]
[arch:09670] [ 5] ./a.out[0x408c19]
[arch:09670] *** End of error message *** 
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 9670 on node arch exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

here's the code

#include <opencv2/opencv.hpp>
#include <mpi.h>

int main(int argc, char **argv) {
    cv::Mat_<uint> img(640,480);
    cv::Mat_<uint> gray(640,480);
    cv::VideoCapture cam(0);

    int rank, nproc, j=0;

    MPI_Status status;

    MPI_Init(&argc, &argv);

    // MPI datatype for 8UC3 image
    MPI_Datatype mat_8uc3;
    MPI_Type_contiguous(sizeof(img), MPI_BYTE, &mat_8uc3);
    MPI_Type_commit(&mat_8uc3);

    // MPI datatype for 8UC1 image
    MPI_Datatype mat_8uc1;
    MPI_Type_contiguous(sizeof(gray), MPI_BYTE, &mat_8uc1);
    MPI_Type_commit(&mat_8uc1);

    MPI_Comm_size(MPI_COMM_WORLD, &nproc); // number of processes
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);  // rank of the current process

    /*
     * Thread 0 captures the image from camera
     * and sends the image to process 1 for processing
     * thread 1 converts the image to grayscale and
     * displays the image
     */

    if (rank == 0) {
        // capture the image and send to thread 1
        while (1) { 
            cam >> img;
            cv::imshow("proc 0", img);
            MPI_Send(&img, 1, mat_8uc3, 1, j, MPI_COMM_WORLD);
            cv::waitKey(40);
            j++;
        }
    }
    else if (rank == 1) {
        // receive the image, convert to grayscale and display
        while (1) {
            MPI_Recv(&img, 1, mat_8uc3, 0, j, MPI_COMM_WORLD, &status);
            cv::cvtColor(img, gray, cv::COLOR_BGR2GRAY);
            cv::imshow("proc 1", gray);
            cv::waitKey(20);
            j++;
        }
    }

    MPI_Finalize();

    return 0;
}

Can someone point out where am I going wrong

Thanks

EDIT: (after user0815's answer)

On making the suggested changes the problem Device or resource busy is resolved but the program still gives a segfault.

[arch:01080] *** Process received signal *** 
[arch:01080] Signal: Segmentation fault (11)
[arch:01080] Signal code: Address not mapped (1) 
[arch:01080] Failing at address: 0x16bbf80
[arch:01080] [ 0] /usr/lib/libpthread.so.0(+0x10740)[0x7fea97322740]
[arch:01080] [ 1] /usr/lib/libopencv_core.so.2.4(_ZNK2cv11_InputArray6getMatEi+0x203)[0x7fea9b302113]
[arch:01080] [ 2] /usr/lib/libopencv_imgproc.so.2.4(_ZN2cv8cvtColorERKNS_11_InputArrayERKNS_12_OutputArrayEii+0x50)[0x7fea9a29bde0]
[arch:01080] [ 3] ./a.out[0x408fc3]
[arch:01080] [ 4] /usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7fea9605f800]
[arch:01080] [ 5] ./a.out[0x408c79]
[arch:01080] *** End of error message *** 
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 1080 on node arch exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

回答1:

Currently each process tries to open the camera. That is very likely to cause problems. Try to move the opening into the root specific section like so:

int main(int argc, char **argv) {
  cv::Mat_<uint> img(640,480);
  cv::Mat_<uint> gray(640,480);
  cv::VideoCapture cam;

  /* ... */
  if (rank == 0) {
    cam.open(0);
    /* ... */
  }
  /* ... */
}

Update:

I think the problem with your code is, that you can't simply transfer objects with MPI_Send. Also is the sizeof operator not valid on objects in general. If you want to transfer an object, you need to transfer the underlying data.

You could achieve this in your case by sending img.data with a size of img.rows * img.cols * sizeof(uint). Then you can also use MPI_BYTE as data type and no custom types are required.

Some details about the internal structure of cv::Mat_ can be found here.

回答2:

As noticed by @user0851 , in your code, all processes try to open the camera and the opening of the camera can be perform by the root process alone.

The Mat object of openCV is quite complex and defining the corresponding MPI_Datatype may be complex too. Instead, sending the array of pixels img.data is much easier. Here is a little piece of code demonstrating how it could be done. It is compiled by mpiCC main.cpp -o main -lopencv_highgui -lopencv_imgproc -lopencv_core and run by mpirun -np 2 main

#include <opencv2/opencv.hpp>
#include <mpi.h>

using namespace cv;

int main(int argc, char **argv) {
    Mat img;
    Mat gray;


    int rank, nproc, j=0;

    size_t total;
    size_t elemsize;
    int sizes[3];

    MPI_Status status;

    MPI_Init(&argc, &argv);

    MPI_Comm_size(MPI_COMM_WORLD, &nproc); // number of processes
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);  // rank of the current process

    /*
     * Thread 0 captures the image from camera
     * and sends the image to process 1 for processing
     * thread 1 converts the image to grayscale and
     * displays the image
     */

    if (rank == 0) {
        VideoCapture cam(0);
        if(!cam.isOpened()){
            fprintf(stderr,"unable to open camera.\n");
            exit(1);
        }
        // capture the image and send to thread 1
        while (1) { 
            cam >> img;
            cv::imshow("proc 0", img);

            if(j==0){
                sizes[2]=img.elemSize();
                Size s = img.size();
                sizes[0] = s.height;
                sizes[1] = s.width;
                MPI_Send( sizes, 3, MPI_INT, 1,0,   MPI_COMM_WORLD);
            }
            MPI_Send( img.data, sizes[0]*sizes[1]*3, MPI_CHAR,1,1, MPI_COMM_WORLD);
            cv::waitKey(40);
            j++;
        }
    }
    else if (rank == 1) {
        // receive the image, convert to grayscale and display
        while (1) {
            if(j==0){
                MPI_Recv( sizes,3, MPI_INT,0,0, MPI_COMM_WORLD,&status);
                img.create(sizes[0],sizes[1],CV_8UC3);
            }
            MPI_Recv( img.data, sizes[0]*sizes[1]*3, MPI_CHAR,0,1, MPI_COMM_WORLD,&status);
            cv::cvtColor(img, gray, cv::COLOR_BGR2GRAY);
            cv::imshow("proc 1", gray);
            cv::waitKey(20);
            j++;
        }
    }

    MPI_Finalize();

    return 0;
}

回答3:

Mat object is only a header structure that points to the memory that stores the image. So you have some problems: First, you create a Mat object sized 640X640 and than you read from the camera into that object. But Mat is just a header it is not a pointer to the data, The Mat object can be now of any width and height.

Second, sizeof(Mat) does not return the amount of memory allocated to the image, only the amount of memory the Mat object itself. The amount of memory needed for the image is Mat.total()*Mat.elemSize()