Making a video with opencv and ffmpeg. How to find

Question

I have a webcam video recorder program built with python, opencv and ffmpeg

It works ok except that the color of the video is more blue than the reality. The problem seems to come from color format of images.

It seems that OpenCv is giving BGR images and ffmpeg+libx264 is expecting YUV420p. I've read that YUV420p correspond to YCbCr.

opencv has no conversion from BGR to YCbCr. It only has a conversion to YCrCb.

I have made some searchs and tried different alternatives to try converting opencv image to something that could be ok for ffmpeg+libx264. None is working. At this point, I am a bit lost and I would appreciate any pointer that could help me to fix this color issue.

Answer 1

Have you tried switching the Cb/Cr channels in OpenCV using split and merge ?

Answer 2

Checked the conversion formulas present in: http://en.wikipedia.org/wiki/YCbCr?

Answer 3

You are right, the default pixel format of OpenCV is BGR.

The equivalent format on the ffmpeg side would be BGR24, so you don't need to convert it to YUV420p if you don't want to.

This post shows how to use a python application to capture frames from the webcam and write the frames to stdout. The purpose is to invoke this app on the cmd-line and pipe the result directly to the ffmpeg application, which stores the frames on the disk. Quite clever indeed!

capture.py:

import cv, sys

cap = cv.CaptureFromCAM(0)
if not cap:
    sys.stdout.write("failed CaptureFromCAM")

while True :
    if not cv.GrabFrame(cap) : 
        break

    frame = cv.RetrieveFrame(cap)
    sys.stdout.write( frame.tostring() )

And the command to be executed on the shell is:

python capture.py | ffmpeg -f rawvideo -pix_fmt bgr24 -s 640x480 -r 30 -i - -an -f avi -r 30 foo.avi

Where:

• -r gives the frame rate coming off the camera
• -an says "don't encode audio"

I tested this solution on my Mac OS X with OpenCV 2.4.2.

EDIT:

In case you haven't tried to record from the camera and use OpenCV to write the video to an mp4 file on the disk, here we go:

import cv, sys

cap = cv.CaptureFromCAM(0)   # 0 is for /dev/video0
if not cap:
    sys.stdout.write("!!! Failed CaptureFromCAM")
    sys.exit(1)

frame = cv.RetrieveFrame(cap)
if not frame: 
    sys.stdout.write("!!! Failed to retrieve first frame")
    sys.exit(1)

# Unfortunately, the following instruction returns 0
#fps = cv.GetCaptureProperty(cap, cv.CV_CAP_PROP_FPS)
fps = 25.0      # so we need to hardcode the FPS
print "Recording at: ", fps, " fps"  

frame_size = cv.GetSize(frame)
print "Video size: ", frame_size  

writer = cv.CreateVideoWriter("out.mp4", cv.CV_FOURCC('F', 'M', 'P', '4'), fps, frame_size, True)
if not writer:
    sys.stdout.write("!!! Error in creating video writer")
    sys.exit(1)


while True :
    if not cv.GrabFrame(cap) : 
        break
    frame = cv.RetrieveFrame(cap)
    cv.WriteFrame(writer, frame)

cv.ReleaseVideoWriter(writer)
cv.ReleaseCapture(cap)

I've tested this with Python 2.7 on Mac OS X and OpenCV 2.4.2.

Answer 4

The libx264 codec is able to process BGR images. No need to use any conversion to YCbCr. NO need to give a spcific pix_ftm to ffmpeg. I was using RGB and it was causing the blueish effect on the video.

The solution was simply to use the original image retuned by the camera without any conversion. :)

I tried this in my previous investigation and it was crashing the app. The solution is to copy the frame returned by the camera.

    frame = opencv.QueryFrame(camera)
    if not frame:
        return None, None

    # RGB : use this one for displaying on the screen
    im_rgb = opencv.CreateImage(self.size,  opencv.IPL_DEPTH_8U, 3)
    opencv.CvtColor(frame, im_rgb, opencv.CV_BGR2RGB)

    # BGR : Use this one for the video
    im_bgr = opencv.CreateImage(self.size,  opencv.IPL_DEPTH_8U, 3)
    opencv.Copy(frame, im_bgr)

    return im_rgb, im_bgr

Answer 5

I've already answered this here. But my VidGear Python Library automates the whole process of pipelining OpenCV frames into FFmpeg and also robustly handles the format conversion. Here's a basic python example:

# import libraries
from vidgear.gears import WriteGear
import cv2

output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer

stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device

writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4' 

# infinite loop
while True:

    (grabbed, frame) = stream.read()
    # read frames

    # check if frame empty
    if not is grabbed:
        #if True break the infinite loop
        break


    # {do something with frame here}
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # write a modified frame to writer
        writer.write(gray) 

        # Show output window
    cv2.imshow("Output Frame", frame)

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

cv2.destroyAllWindows()
# close output window

stream.release()
# safely close video stream
writer.close()
# safely close writer

Source: https://github.com/abhiTronix/vidgear/wiki/Compression-Mode:-FFmpeg#2-writegear-classcompression-mode-with-opencv-directly

You can check out full VidGear Docs for more advanced applications and exciting features.

Hope that helps!