Media Foundation onReadSample wrong size of return

I am working on translating a capture library from DirectShow to MediaFoundation. The capture library seemed to work quite well but I face a problem with an integrated webcam on a tablet running Windows 8 32 bit.

When enumerating the capture format (as explained in Media Foundation documentation), I got the following supported format for the camera:

0 : MFVideoFormat_NV12, resolution : 448x252, framerate : 30000x1001
1 : MFVideoFormat_YUY2, resolution : 448x252, framerate : 30000x1001
2 : MFVideoFormat_NV12, resolution : 640x360, framerate : 30000x1001
3 : MFVideoFormat_YUY2, resolution : 640x360, framerate : 30000x1001
4 : MFVideoFormat_NV12, resolution : 640x480, framerate : 30000x1001
5 : MFVideoFormat_YUY2, resolution : 640x480, framerate : 30000x1001

I then set the capture format, in this case the one at index 5, using the following function, as described in the example:

hr = pHandler->SetCurrentMediaType(pType);

This function executed without error. The camera should thus be configured to capture in YUY2 with a resolution of 640*480.

In the onReadSample callback, I should receive a sample with a buffer of size :

640 * 480 * sizeof(unsigned char) * 2 =  614400 //YUY2 is encoded on 2 bytes

However, I got a sample with a buffer of size 169344. Here below is a part of the callback function.

HRESULT SourceReader::OnReadSample(
    HRESULT hrStatus,
    DWORD dwStreamIndex,
    DWORD dwStreamFlags,
    LONGLONG llTimeStamp,
    IMFSample *pSample      // Can be NULL
    )
{
    EnterCriticalSection(&m_critsec);

    if (pSample)
    {
        DWORD expectedBufferSize = 640*480*1*2; // = 614400 (hard code for the example)

        IMFMediaBuffer* buffer = NULL;
        hr = pSample->ConvertToContiguousBuffer(&buffer);
        if (FAILED(hr))
        {
            //...
            goto done;
        }

        DWORD byteLength = 0;
        BYTE* pixels = NULL;
        hr = buffer->Lock(&pixels, NULL, &byteLength);

        //byteLength is 169344 instead of 614400 
        if (byteLength > 0 && byteLength == expectedBufferSize)
        {
            //do someting with the image, but never comes here because byteLength is wrong
        }
        //...

Any advice why I get a sample of size 169344 ?

Thanks in advance

Thanks Mgetz for your answer.

I checked the value of MF_MT_INTERLACE_MODE of the media type and it appears that the video stream contains progressive frames. The value of MF_MT_INTERLACE_MODE returns MFVideoInterlace_Progressive.

hr = pHandler->SetCurrentMediaType(m_pType);
if(FAILED(hr)){
    //
}
else
{
    //get info about interlacing
    UINT32 interlaceFormat = MFVideoInterlace_Unknown;
    m_pType->GetUINT32(MF_MT_INTERLACE_MODE, &interlaceFormat);

    //...

So the video stream is not interlaced. I checked again in the onReadSample the value of MFSampleExtension_Interlaced to see if the sample is interlaced or not and it appears that the sample is interlaced.

if (pSample && m_bCapture)
{
    //check if interlaced 
    UINT32 isSampleInterlaced = 0;
    pSample->GetUINT32(MFSampleExtension_Interlaced, &isSampleInterlaced);
    if(isSampleInterlaced)
    {
        //enters here
    }

How it is possible that the stream is progressive and that the sample is interlaced? I double checked the value of MF_MT_INTERLACE_MODE in the onReadSample callback as well and it still gives me the value MFT_INPUT_STREAM_WHOLE_SAMPLES.

Concerning your first suggestion, I didn't way to force the flag MFT_INPUT_STREAM_WHOLE_SAMPLES on the input stream.

Thanks in advance

I still face the issue and I am now investigating on the different streams available.

According to the documentation, each media source provides a presentation descriptor from which we can get the streams available. To get the presentation descriptor, we have to call:

HRESULT hr = pSource->CreatePresentationDescriptor(&pPD);

I then request the streams available using the IMFPresentationDescriptor::GetStreamDescriptorCount function:

DWORD nbrStream;
pPD->GetStreamDescriptorCount(&nbrStream);

When requesting this information on the frontal webcam on an ACER tablet running windows 8, I got that three streams are available. I looped over these streams, requested their MediaTypeHandler and checked the MajorType. The three streams have for major type : MFMediaType_Video, so all the streams are video streams. When listing the media type available on the different streams, I got that all the streams support capture at 640x480. (some of the streams have more available media types).

I tested to select each of the different streams and the appropriate format type (the framework did not return any error), but I still do not receive the correct sample in the callback function...

Any advice to progress on the issue?

Finally found the issue: I had to set the media type on the source reader directly, using SourceReader->SetCurrentMediaType(..). That did the trick!

Thanks for your help!

回答1:

Without knowing what the input media type descriptor is we can largely only speculate, but the most likely answer is you are saying you can handle the stream even though MFT_INPUT_STREAM_WHOLE_SAMPLES is not set on the input stream.

The next most likely cause is interlacing in which case each frame would be complete but not full resolution which you are assuming. Regardless you should verify the ENTIRE media type descriptor before accepting it.

回答2:

Finally found the issue: I had to set the media type on the source reader directly, using SourceReader->SetCurrentMediaType(..). That did the trick!

Thanks for your help!