Audio Sync problems using DirectShow.NET

Question

I have started a thread on this at DirectShow.NET's forum, here is the link http://sourceforge.net/projects/directshownet/forums/forum/460697/topic/5194414/index/page/1 but unfortunately the problem still persists...

I have an application that captures video from a webcam and audio from the microphone and save it to a file, for some reason the audio and video are never in-sync, i tried the following: 1. Started with ffdshow encoder and changed to AVI Mux - problem persists, audio is delayed and at the end of the video the picture remains frozen and the audio continues 2. Changed from AVI Mux to WM ASF Writer - video is frozen at the beginning (2 seconds) and rest of video is in-sync (but the two first seconds are not usable) 3. create SampleGrabber that prints the timestamp for both audio and video - saw that the audio timestamp is 500ms earlier but I have no idea what to do with this fact... 4. tried manually setting the ReferenceClock to one of the capture filters (audio/video) but both won't cast to IReferenceClock 5. Created a SystemClock and set it has the ReferenceClock - made no difference 6. Set SyncUsingStreamOffset(true) on the grap - timestamps are much closer now but the final result is the same 7. Tried saving the audio and video to two different files and used VirtualDub to see if they match, they still dont...

Oh i forgot to mention I also tried building the graph in GraphEditPlus but the problem still remains, here's a link to the graph: http://www2.picturepush.com/photo/a/8030745/img/8030745.png

Currently I am testing all my changes on the CapWMV sample from DirectShow.NET's samples.

Please any advice would be highly appreciated, I am hopeless :/

Thanks,

Eran.

Update:

It seems there's a constant 500ms gap between the audio and video, if I use virtualDub to delay the audio by 500ms it looks fine, how can set this in the graph?

Answer 1

Just wanted to add the solution for my situation, maybe it will help someone.

I was trying to record video from a webcam together with audio from a microphone, video is HD (1080p) so I wanted to save an AVI file encoded in MPEG4, so I used ffshow-tryous (free Mpeg4 encoder) together with an Avi Mux Filter, the problem was that some (well most of them :) ) of my videos had sync issues.

What I discovered was that Avi Mux does not handle synchronization, it assumes the data arrives at the appropriate time (written here - http://msdn.microsoft.com/en-us/library/dd407208(v=vs.85).aspx), so I tried using WMAsfWriter which does handle synchronization and it worked fine (The 2 seconds freeze I mentioned above was just a glitch with VLC Player) but it doesn't work good with high resolutions and I had trouble using it with custom profiles (filters won't get connected).

I also tried Roman's suggestion and although the links were very interesting and promising (I really recommend reading them - can't give +1 to a post yet...) it just didn't made any difference :/

My final solution was to give up on MPEG4 and just use MPEG2, I switched from Avi Mux to Microsoft MPEG2 Encoder which works great, should have thought about that long time ago :)

Hopefully this will help someone else.

Thanks,

Eran.

Answer 2

You are having latency on the audio stream equal to size of capture buffer. That is, you obtain the full buffer which started being captured 0.5 seconds away. You need to use smaller buffers and/or apply offset on the buffers to adjust the latency.

See:

• Minimizing audio capture latency in DirectShow
• How to eliminate 1 second delay in DirectShow filter chain? (Using Delphi and DSPACK)

IAMBufferNegotiation is the keyword.

Answer 3

I had the same problem rendering video from WMV to AVI using Xvid MPEG-4 decoder.

My final solution without giving up MPEG-4 was to configure the AviMuxer setting ConfigAviMux::SetMasterStream property

As explained in the Capturing Video to an AVI File article from MSDN configuration:

If you are capturing audio and video from two separate devices, it is a good idea to make the audio stream the master stream. This helps to prevent drift between the two streams, because the AVI Mux filter adjust the playback rate on the video stream to match the audio stream.

Example Code :

IConfigAviMux _filterAVIMuxerCfg = (IConfigAviMux)_filterAVIMuxer;
_filterAVIMuxerCfg.SetMasterStream(0); // I've add first audio ;)