Capturing video out of an OpenGL window in Windows

Question

I am supposed to provide my users a really simple way of capturing video clips out of my OpenGL application's main window. I am thinking of adding buttons and/or keyboard shortcuts for starting and stopping the capture; when starting, I could ask for a filename and other options, if any. It has to run in Windows (XP/Vista), but I also wouldn't like to close the Linux door which I've so far been able to keep open.

The application uses OpenGL fragment and shader programs, the effects due to which I absolutely need to have in the eventual videos.

It looks to me like there might be even several different approaches that could potentially fulfill my requirements (but I don't really know where I should start):

• An encoding library with functions like startRecording(filename), stopRecording, and captureFrame. I could call captureFrame() after every frame rendered (or every second/third/whatever). If doing so makes my program run slower, it's not really a problem.

• A standalone external program that can be programmatically controlled from my application. After all, a standalone program that can not be controlled almost does what I need... But as said, it should be really simple for the users to operate, and I would appreciate seamlessness as well; my application typically runs full-screen. Additionally, it should be possible to distribute as part of the installation package for my application, which I currently prepare using NSIS.

• Use the Windows API to capture screenshots frame-by-frame, then employ (for example) one of the libraries mentioned here. It seems to be easy enough to find examples of how to capture screenshots in Windows; however, I would love a solution which doesn't really force me to get my hands super-dirty on the WinAPI level.

• Use OpenGL to render into an offscreen target, then use a library to produce the video. I don't know if this is even possible, and I'm afraid it might not be the path of least pain anyway. In particular, I would not like the actual rendering to take a different execution path depending on whether video is being captured or not. Additionally, I would avoid anything that might decrease the frame rate in the normal, non-capture mode.

If the solution were free in either sense of the word, then that would be great, but it's not really an absolute requirement. In general, the less bloat there is, the better. On the other hand, for reasons beyond this question, I cannot link in any GPL-only code, unfortunately.

Regarding the file format, I cannot expect my users to start googling for any codecs, but as long as also displaying the videos is easy enough for a basic-level Windows user, I don't really care what the format is. However, it would be great if it were possible to control the compression quality of the output.

Just to clarify: I don't need to capture video from an external device like camcorder, nor am I really interested in mouse movements, even though getting them does not harm either. There are no requirements regarding audio; the application makes no noise whatsoever.

I write C++ using Visual Studio 2008, for this very application also taking benefit of GLUT and GLUI. I have a solid understanding regarding C++ and linking in libraries and that sort of stuff, but on the other hand OpenGL is quite new for me: so far, I've really only learnt the necessary bits to actually get my job done.

I don't need a solution super-urgently, so feel free to take your time :)

Answer 1

There are two different questions here - how to grab frames from an OpenGL application, and how to turn them into a movie file.

The first question is easy enough; you just grab each frame with glReadPixels() (via a PBO if you need the performance).

The second question is a little harder since the cross-platform solutions (ffmpeg) tend to be GPL'd or LGPL'd. Is LGPL acceptable for your project? The Windows way of doing this (DirectShow) is a bit of a headache to use.

Edit: Since LGPL is ok and you can use ffmpeg, see here for an example of how to encode video.

Answer 2

This does look pretty relevant for merging into an AVI (as suggested by Andrew), however I was really hoping to avoid the LPBITMAPINFOHEADERs etc.

Thanks for the answers, I will report on the success if there is going to be any :)

In the meantime, additional tips for encoding the raw frames from glReadPixels into video clips would be appreciated.

Edit: So far, ffmpeg suggested by Mike F seems to be the way to go. However, I didn't get into the actual implementation yet, but hopefully that will change in the near future!

Answer 3

I had to create a demo project of recording an OpenGL rendering into a video. I used glReadPixels to get the pixel data and created the video with OpenCV's cvWriteFrame. OpenCV lets you write in divx or even x264/vp8(with ffmpeg compiled in).

I have a more detailed writeup on my blog post along with a sample project. http://tommy.chheng.com/2013/09/09/encode-opengl-to-video-with-opencv/

Answer 4

The easiest option is going to be saving each rendered frame from within your app and then merging them into an AVI. When you have the AVI there are many libraries available that can convert it into a more optimal format, or possibly skip the AVI step altogether.

In terms of getting each frame, you could accomplish this either by rendering into an offscreen texture as you suggest or using the backbuffer directly as a source if your hardware supports this. Doing either of these (and saving each frame) is going to be difficult without a heavy penalty on framerate.

Providing your application is deterministic you could "record" the users actions as a series of inputs and then have an export mode that sequentially renders these to an offscreen surface to generate the AVI.