Very Slow Processing of my Opencv Application

Question

I am building an OpenCV application which captures a video from camera and overlays it on another video after removing the background.

I am not able to achieve a reasonable speed as it is playing the output at about 1 fps, whereas my background removal is working at 3fps.

Is there a way to display the background video at its normal speed and overlay the processed video at 3fps ?

I tried commenting out my code and I realized that the problem lies majorly with the Rendering part itself. I tried displaying the video along with my web cam feed and I noticed that there is a drop in the actual fps and the fps of video when displayed with openCV.

here is the sample code:

 void main()
{
    CvCapture* capture, *Vcap;
    capture = cvCaptureFromCAM(0);
    if(!capture)
    {
        printf("Video Load Error");
    }

    Vcap = cvCaptureFromAVI("bgDemo.mp4");
    //printf("\nEntered BGR");
    if(!Vcap)
    {
        printf("Video Load Error");
    }

    while(1)
    {

        IplImage* src = cvQueryFrame(Vcap);
        if(!src)
        {
            Vcap = cvCaptureFromAVI("bgDemo.mp4");
            continue;
        }
        IplImage* bck1 = cvCreateImage(cvGetSize(src),8,3);
        cvResize(src,bck1,CV_INTER_LINEAR);

        cvShowImage("BCK",bck1);
        cvWaitKey(1);
    }
}

Answer 1

The main problem is that you are allocating a new image at every iteration of the loop without releasing it at the end of the loop. In other words, you have a beautiful memory leak.

A better approach is to simply grab a frame of the video before the loop starts. This will let you create bck1 with the right size just once.

There are other problems with your code, I'm sharing a fixed version below, make sure you pay attention to every line of code to see what changed. I haven't had time to test it, but I'm sure you'll figure it out:

int main()
{
    // I know what you are doing, just one capture interface is enough
    CvCapture* capture = NULL; 

    capture = cvCaptureFromCAM(0);
    if(!capture)
    {
        printf("Ooops! Camera Error");            
    }

    capture = cvCaptureFromAVI("bgDemo.mp4");
    if(!capture)
    {
        printf("Ooops! Video Error");
        // if it failed here, it means both methods for loading a video stream failed.
        // It makes no sense to let the application continue, so we return.
        return -1; 
    }

    // Retrieve a single frame from the camera
    IplImage* src = cvQueryFrame(capture);
    if(!src)
    {
        printf("Ooops! #1 cvQueryFrame Error");
        return -1; 
    }

    // Now we can create our backup image with the right dimensions.
    IplImage* bck1 = cvCreateImage(cvGetSize(src),src->depth, src->nChannels);
    if(!bck1)
    {
        printf("Ooops! cvCreateImage Error");
        return -1; 
    }

    while(1)
    {
        src = cvQueryFrame(capture);
        if(!src)
        {
             printf("Ooops! #2 cvQueryFrame Error");  
             break; 
        }

        cvResize(src, bck1, CV_INTER_LINEAR);

        cvShowImage("BCK",bck1);
        cvWaitKey(10);
    }

    cvReleaseImage( &bck1 ); // free manually allocated resource

    return 0;
}

These fixes should speed up your application considerably.