Record video with Accord.net (AForge)

Question

I used Accord.net (AForge) for connect to the webcam and record video But stored videos is slow motion ... this my project :

    using AForge.Video;
using AForge.Video.DirectShow;
using AForge.Video.FFMPEG;
using System;
using System.Drawing;
using System.IO;
using System.Threading;
using System.Windows.Forms;

namespace CaptureWebcam
{
    public partial class Form1 : Form
    {
        private VideoCaptureDeviceForm captureDevice;
        private string path = Path.GetDirectoryName(Application.ExecutablePath) + @"\Videos\";
        private FilterInfoCollection videoDevice;
        private VideoCaptureDevice videoSource;
        private VideoFileWriter FileWriter = new VideoFileWriter();
        private Bitmap video;
        bool isRecord = false;

        public Form1()
        {
            InitializeComponent();
        }

        private void videoSource_NewFrame(object sender, NewFrameEventArgs eventArgs)
        {
            video = (Bitmap)eventArgs.Frame.Clone();
            pictureBox1.Image = (Bitmap)eventArgs.Frame.Clone();
        }

        private void btnStartCam_Click(object sender, EventArgs e)
        {
            videoDevice = new FilterInfoCollection(FilterCategory.VideoInputDevice);
            captureDevice = new VideoCaptureDeviceForm();
            if (captureDevice.ShowDialog(this) == DialogResult.OK)
            {
                videoSource = captureDevice.VideoDevice;
                videoSource = captureDevice.VideoDevice;
                videoSource.NewFrame += new NewFrameEventHandler(videoSource_NewFrame);
                videoSource.Start();
                timer1.Enabled = true;
            }
            //videoSource.DisplayPropertyPage(IntPtr.Zero);
        }

        private Thread workerThread = null;
        private bool stopProcess = false;

        private void recordLiveCam()
        {
            if (!stopProcess)
            {
                while (isRecord)
                {
                    FileWriter.WriteVideoFrame(video);
                }
                FileWriter.Close();
            }
            else
            {
                workerThread.Abort();
            }            
        }

        private void btnRecord_Click(object sender, EventArgs e)
        {
            //try
            //{
            isRecord = true;
            if (!Directory.Exists(path))
            {
                Directory.CreateDirectory(path);
            }
            int h = captureDevice.VideoDevice.VideoResolution.FrameSize.Height;
            int w = captureDevice.VideoDevice.VideoResolution.FrameSize.Width;
            FileWriter.Open(path + "recorded at " + DateTime.Now.ToString("HH-mm-ss") + ".mp4", w, h, 25, VideoCodec.MPEG4);
            stopProcess = false;
            workerThread = new Thread(new ThreadStart(recordLiveCam));
            workerThread.Start();

            //}
            //catch (Exception ex)
            //{
            //    MessageBox.Show(ex.Message);
            //}
        }

        private void Form1_Load(object sender, EventArgs e)
        {

        }

        private void btnStopRecord_Click(object sender, EventArgs e)
        {
            stopProcess = true;
            isRecord = false;
            FileWriter.Close();
            workerThread.Abort();
            video = null;

        }

        private void btnStopCam_Click(object sender, EventArgs e)
        {
            try
            {
                if (videoSource.IsRunning)
                {
                    videoSource.SignalToStop();
                    videoSource.Stop();                  
                    pictureBox1.Image = null;
                    pictureBox1.Invalidate();
                    if (FileWriter.IsOpen)
                    {
                        FileWriter.Close();
                        video = null;
                    }
                }
                else
                    return;
            }
            catch
            {
                videoSource.Stop();
                FileWriter.Close();
                video = null;
            }
        }

        float fts = 0.0f;
        private void timer1_Tick(object sender, EventArgs e)
        {
            fts = videoSource.FramesReceived;
            label1.Text = "Frame Rate : " + fts.ToString("F2") + " fps";
        }
    }
}

And when click the btnStopRecord following error :

Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

Answer 1

"Slow motion" can have multiple reasons. First you need to know if the actual framerate, at which the NewFrames get produces is so slow (how many HZ then?) or if the CPU is too busy. Or the graphics card maybe.

For everything DirectShow related I strongly suggest using GraphEdit and AMCap to see what the real capabilites of the device are. AForge was prone to oversampling and I would not wonder if it's similar in Accord.

Also, you can always rely on good old processexplorer to see if the load (if any) is caused by SYSTEM or INTERRUPRS or is actually produced by your executable.

Another thing that happened to me was PictureBox. VideoSourcePlayer is far better and I wound up doing my own blitting.

It is worth mentioning these three optimizations:

• DO NOT USE Bitmap.GetPixel!
• Use videoSource.DisplayPinPropertyPage(IntPtr.Zero)
• Check the color space of the videostream

Bitmap.GetPixel

The problem with GetPixel is, that it's really slow as it has to unmanage/manage a lot for every single pixel. It is fine as along as you only get a small a mount of calls and use it on a loaded bitmap. It is definately the wrong approach if you want to use it on all pixelx every couple of milliseconds. The CPU load from this phenomenon is associated with your application and will show as such in process explorer. In the end I wrote my own pixel routines starting here: Work with bitmap faster with Csharp If you just want a kernel or something more usual than I needed, implement a custom Filter.

VideoSource.DisplayPinPropertyPage

If you use the AForge-buit-in mechanism for choosing the resolution and initiating the video stream, you cannot set the Framerate (I do not consider this a bug in AForge). So AForge always chooses the highest framerate. If you circumvent videoSource.VideoCapabilities and directly show the native device configuration dialog (it's formally called "Video Capture Pin Properties Dialog"). There you can set all the parameters manually and the dialog will fill in the appropriate framerate. This way, you will get the "real" (hardware) refresh rate in your callbacks. The CPU load of this oversampling happens in the SYSTEM process.

Color space conversion

In my case, the camera supports three output formats: YUY2, MJPG and RGB24. Depending on what format you use, the image is transformed by AForge into RGB (I think actually it's ARGB32). The rendered DirectShow graphs for all three formats are different and RGB obviously uses much less CPU than the others, as the color conversion is quite trivial. The load from this conversion also shows up in the SYSTEM process and not in your applications exe.

Answer 2

You can try to put some timers to understand which operation is slowing down the process.(Stopwatch class will be perfect)
I don't know the format of your Frame but usually the conversion to Bitmap is the bottle neck, AForge is quite fast. Furthermore you can pass a time stamp to WriteVideoFrame method, then the frame rate indicated in Open is just for replaying the video. Aforge will ordinate the frames in the right order with the right time.
That's according to my experience. Hope it can help.