Media Foundation MFTransform to convert MFSample f

Question

Do anyone know what MFTransform should I use to convert a 'MJPG' MFSample to 'YUY2' or 'RGB24'?

Any tip will be very appreciated. Thanks

Answer 1

Since the number of visits to this post was noticeable I'm going to answer my own question.

The trick is to enum all the transformations between MJPG and YUY2 since it seems there isn't direct transformation between MJPG and RGB32. To transform the image between YUY2 and RBG32 use the color converter DSP: http://msdn.microsoft.com/en-us/library/windows/desktop/ff819079%28v=vs.85%29.aspx

I use this method to fetch samples from a 1080p webcam and then decoding MJPG to YUY2 and then decoding YUY2 to RGB32 and then loading a OpenGL RGB32 texture, and then displaying it. It is done at 30fps with a core 2 duo, and a Radeon HD5650. When doing come computation with the images with OpenCL (several convolutions) it drops to 15 fps.

Code to create a MJPG to YUY2 transform:

MFT_REGISTER_TYPE_INFO inputFilter = { MFMediaType_Video, MFVideoFormat_MJPG };
MFT_REGISTER_TYPE_INFO outputFilter = { MFMediaType_Video, MFVideoFormat_YUY2 };
UINT32 unFlags = MFT_ENUM_FLAG_SYNCMFT | MFT_ENUM_FLAG_LOCALMFT | MFT_ENUM_FLAG_SORTANDFILTER;

HRESULT r = MFTEnumEx(MFT_CATEGORY_VIDEO_DECODER, unFlags, &inputFilter, &outputFilter, &ppActivate, &numDecodersMJPG);
if (FAILED(r)) throw gcnew Exception("");
if (numDecodersMJPG < 1) throw gcnew Exception("");

// Activate transform
IMFTransform *pMPEG4 = NULL;
r = ppActivate[0]->ActivateObject(__uuidof(IMFTransform), (void**)&pMPEG4);
if (FAILED(r)) throw gcnew Exception("No se pudo crear el decodificador MJPG.");

The next part is to use the decoder to decode the compressed sample (first from MJPG to YUY2 and then from YUY2 to RGB32). It is explained in: http://msdn.microsoft.com/en-us/library/windows/desktop/aa965264%28v=vs.85%29.aspx

Or:

        MFT_OUTPUT_STREAM_INFO osi;

    HRESULT r = pDecoder->ProcessMessage(MFT_MESSAGE_NOTIFY_BEGIN_STREAMING, 0);
    if (FAILED(r)) throw gcnew Exception("");

    r = pDecoder->ProcessInput(0, sample, 0);
    if (FAILED(r)) throw gcnew Exception("");

    r = pDecoder->GetOutputStreamInfo(0, &osi);
    if (FAILED(r)) throw gcnew Exception("");

    DWORD status = 0;
    r = pDecoder->GetOutputStatus(&status);
    if (FAILED(r)) throw gcnew Exception("");
    if (status = MFT_OUTPUT_STATUS_SAMPLE_READY) {
    }

    // Use your own CreateSample function
    IMFSample *outputSample = CreateSample(osi.cbSize);

    DWORD outStatus = 0;
    MFT_OUTPUT_DATA_BUFFER odf;
    odf.dwStreamID = 0;
    odf.pSample = outputSample;
    odf.dwStatus = 0;
    odf.pEvents = NULL;
    r = pDecoder->ProcessOutput(0, 1, &odf, &outStatus);
    if (r != MF_E_TRANSFORM_NEED_MORE_INPUT && FAILED(r)) {
        outputSample->Release();
        throw gcnew Exception("");
    }

    r = pDecoder->ProcessMessage(MFT_MESSAGE_NOTIFY_END_OF_STREAM, 0);
    if (FAILED(r)) {
        outputSample->Release();
        throw gcnew Exception("");
    }

    r = pDecoder->ProcessMessage(MFT_MESSAGE_COMMAND_DRAIN, 0);
    if (FAILED(r)) {
        outputSample->Release();
        throw gcnew Exception("");
    }

    return outputSample;