C++ OpenCV 3.4 / FFMPEG 3.4.1 VideoWriter and MP4

Question

I'm running on an ARM BeagleBone X-15 Debian machine with Linux Kernel 4.9.35-ti-r44. In my C++ ( Qt 5 ) application, I want to save my cv::Mat frames to an MP4 format video. I have libx264 installed and compiled from scratch both ffmpeg and OpenCv. I can successfully record and view video with MJPEG with the AVI video container type but not MP4. Whenever I try to write an MP4 video, I get run time errors of the form:

OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000020/' ???'

I've tried various Google answers but none have worked.

Relevant code:

cv::Size frameSize = frame.size();

qDebug() << "Initializing Video Recording to save Video file here: " << destinationFileName;

std::string filename = destinationFileName.toStdString();
//int fcc =   CV_FOURCC('M','J','P','G');
//int fcc =   CV_FOURCC('X','2','6','4');
int fcc = CV_FOURCC('X','V','I','D');
int fps =   1;
videoRecorder = new cv::VideoWriter(filename,fcc,fps,frameSize);

...

videoRecorder->write(frame);

I've downloaded and built the latest OpenCV 3.4 but the problem persists. How can I write to an MP4 File with OpenCV's video writer?

When I try the 'X','2','6','4' format I get the error:

    Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
[h264_v4l2m2m @ 0x81042020] Could not find a valid device
[h264_v4l2m2m @ 0x81042020] can't configure encoder
Could not open codec 'h264_v4l2m2m': Unspecified error
Starting

When I try the 'X','V','I','D' format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Starting

When I try the 'M','P','E','G' format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x4745504d/'MPEG' is not supported with codec id 2 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Starting

When I try the 'H','2','6','4' format I get the error:

OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
[h264_v4l2m2m @ 0x7fdde340] Could not find a valid device
[h264_v4l2m2m @ 0x7fdde340] can't configure encoder
Could not open codec 'h264_v4l2m2m': Unspecified error
Starting

When I try the 'M','P','4','V' format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Starting

When I try the 'A','V','C','1' format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x31435641/'AVC1' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'
[h264_v4l2m2m @ 0x810f5f50] Could not find a valid device
[h264_v4l2m2m @ 0x810f5f50] can't configure encoder
Could not open codec 'h264_v4l2m2m': Unspecified error
Starting

When I try the 'D','I','V','X' format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x58564944/'DIVX' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Starting

When I try the 0x21 format I get the error:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x00000021/'!???' is not found (format 'mp4 / MP4 (MPEG-4 Part 14)')'
Starting

Here is my complete class which shows all tried formats:

    #include "downloader.h"

Downloader::Downloader(QString url, QString destinationFile) : downloadUrl(url) , destinationFileName(destinationFile)
{

    didInitializeVideoWriter = false;

    qDebug() << "Initialized Downloader...";

}

Downloader::~Downloader() {

    videoRecorder->release();
    delete videoRecorder;

}

void Downloader::doDownload()
{
    networkManager = new QNetworkAccessManager(this);

    connect(networkManager, SIGNAL(finished(QNetworkReply*)),
            this, SLOT(replyFinished(QNetworkReply*)));

    networkManager->get(QNetworkRequest(QUrl(downloadUrl)));
}

void Downloader::writeFrame(cv::Mat frame) {

    QMutexLocker locker(&videoFrameMutex);

    videoRecorder->write(frame);

}

void Downloader::replyFinished(QNetworkReply *reply)
{

    if(reply->error())
    {
        qDebug() << "ERROR!";
        qDebug() << reply->errorString();
    }
    else
    {
        //qDebug() << reply->header(QNetworkRequest::ContentTypeHeader).toString();
        //qDebug() << reply->header(QNetworkRequest::LastModifiedHeader).toDateTime().toString();
        //qDebug() << reply->header(QNetworkRequest::ContentLengthHeader).toULongLong();
        qDebug() << reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
        qDebug() << reply->attribute(QNetworkRequest::HttpReasonPhraseAttribute).toString();

        QString tempFileName = destinationFileName;
        QString jpegTempFilename = QString("%1").arg(tempFileName.replace("mp4","jpeg"));

        qDebug() << "Overwriting: " << jpegTempFilename;
        QFile *file = new QFile(jpegTempFilename);
        if(file->open(QFile::WriteOnly))
        {
            file->write(reply->readAll());
            file->flush();
            file->close();
        }
        delete file;

        cv::Mat frame = imread(jpegTempFilename.toStdString(), CV_LOAD_IMAGE_COLOR);   // CV_LOAD_IMAGE_COLOR (>0) loads the image in the BGR format
        cv::cvtColor(frame,frame,CV_BGR2RGB);

        // Now lazy load the recorder
        if ( !didInitializeVideoWriter ) {

            cv::Size frameSize = frame.size();

            qDebug() << "Initializing Video Recording to save Video file here: " << destinationFileName;

            std::string filename = destinationFileName.toStdString();
            //int fcc =   CV_FOURCC('M','J','P','G');
            int fcc =   CV_FOURCC('X','2','6','4');
            //int fcc = CV_FOURCC('X','V','I','D');
            //int fcc = CV_FOURCC('M','P','E','G');
            //int fcc = CV_FOURCC('H','2','6','4');
            //int fcc = CV_FOURCC('M','P','4','V');
            //int fcc = CV_FOURCC('A','V','C','1');
            //int fcc = CV_FOURCC('D','I','V','X');
            //int fcc = 0x21;
            //int fcc = 0x00000021;
            int fps =   1;
            videoRecorder = new cv::VideoWriter(filename,fcc,fps,frameSize);

            qDebug() << "Starting";
            frameCounter = 1;
            performanceTimer.start();

            didInitializeVideoWriter = true;

        }

        cv::putText(frame,"[REC]",cv::Point(50,50),5,1,cv::Scalar(0,0,225));

        QFuture<void> backgroundRun = QtConcurrent::run(this, &Downloader::writeFrame, frame);
        //backgroundRun.waitForFinished();

    }

    reply->deleteLater();

    qDebug() << "RequestTimer: " << performanceTimer.elapsed() << frameCounter;

    // Requests Again
    networkManager->get(QNetworkRequest(QUrl(downloadUrl)));

    frameCounter++;

    performanceTimer.restart();


}

Update - I tried 'a','v','c','1' and unfortunately that also does not work:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
[h264_v4l2m2m @ 0x810f6aa0] Could not find a valid device
[h264_v4l2m2m @ 0x810f6aa0] can't configure encoder
Could not open codec 'h264_v4l2m2m': Unspecified error

(qt-downloader:6234): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
OpenCV Error: Unspecified error (GStreamer: cannot link elements
) in CvVideoWriter_GStreamer::open, file /media/usb/opencv/modules/videoio/src/cap_gstreamer.cpp, line 1635
VIDEOIO(cvCreateVideoWriter_GStreamer (filename, fourcc, fps, frameSize, is_color)): raised OpenCV exception:

/media/usb/opencv/modules/videoio/src/cap_gstreamer.cpp:1635: error: (-2) GStreamer: cannot link elements
 in function CvVideoWriter_GStreamer::open

When I try the X264 FOURCC, the mp4 file is 48 bytes and never grows:

Initializing Video Recording to save Video file here:  "/tmp/Garage.mp4"
OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 27 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x31637661/'avc1'

Static Size:

debian@BeagleBoard-X15:/tmp$ ls -lrt Garage.*
-rw-r--r-- 1 debian debian     48 Dec 24 21:13 Garage.mp4
-rw-r--r-- 1 debian debian 100424 Dec 24 21:14 Garage.jpeg
debian@BeagleBoard-X15:/tmp$ hexdump Garage.mp4 
0000000 0000 2000 7466 7079 7369 6d6f 0000 0002
0000010 7369 6d6f 7369 326f 7661 3163 706d 3134
0000020 0000 0800 7266 6565 0000 0000 646d 7461
0000030

Here is my ffmpeg build conf:

debian@BeagleBoard-X15:/tmp$ ffmpeg -buildconf
ffmpeg version N-89524-g74f408cc8e Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --enable-gpl --enable-libx264 --enable-pthreads --enable-static --extra-cflags=-I./x264/include --extra-ldflags=-L./x264/lib --extra-libs=-ldl
  libavutil      56.  6.100 / 56.  6.100
  libavcodec     58.  8.100 / 58.  8.100
  libavformat    58.  3.100 / 58.  3.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  7.100 /  7.  7.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100

  configuration:
    --enable-gpl
    --enable-libx264
    --enable-pthreads
    --enable-static
    --extra-cflags=-I./x264/include
    --extra-ldflags=-L./x264/lib
    --extra-libs=-ldl

Answer 1

Did you tried ust in plain C++11 like :

    using namespace std;
    using namespace cv;

int main(){

    // Create a VideoCapture object and use camera to capture the video
    VideoCapture cap(0);

    // Check if camera opened successfully
    if(!cap.isOpened())
    {
        cout << "Error opening video stream" << endl;
        return -1;
    }

    // Default resolution of the frame is obtained.The default resolution is system dependent.
    int frame_width = cap.get(CV_CAP_PROP_FRAME_WIDTH);
    int frame_height = cap.get(CV_CAP_PROP_FRAME_HEIGHT);

    // Define the codec and create VideoWriter object.The output is stored in 'outcpp.avi' file.
    VideoWriter video("outcpp-.avi",CV_FOURCC('M','J','P','G'),10, Size(frame_width,frame_height));
    while(1)
    {
        Mat frame;

        // Capture frame-by-frame
        cap >> frame;

        // If the frame is empty, break immediately
        if (frame.empty())
            break;

        // Write the frame into the file 'outcpp.avi'
        video.write(frame);

        // Display the resulting frame
        imshow( "Frame", frame );

        // Press  ESC on keyboard to  exit
        char c = (char)waitKey(1);
        if( c == 27 )
            break;
    }

    // When everything done, release the video capture and write object
    cap.release();
    video.release();

    // Closes all the windows
    destroyAllWindows();
    return 0;
}

If it is working you have the same problem that we have :) QT has problem with the videowriter Opencv I assume. Welcome to the club.

On the other hand did you find a solution ?

Answer 2

I had similar issue with you, but running on MacOS. I solved it by re-installing X264 library to the latest version, and then re-install FFMPEG with ./configure --enable-libx264 --enable-gpl --enable-avresample --enable-shared to link the new X264 to FFMPEG properly. Then, re-install OpenCV (I am using version 3.4.4) to link the new FFMPEG. When building the OpenCV, make sure that WITH FFMPEG is ON otherwise built-in OpenCV encoder will be used, which will give you more limited options.