How to use hardware video scalers?

问题:

Modern graphics cards have hardware video scalers, for example as part of AMD Avivo, NVIDIA PureVideo or Intel ClearVideo. For example, AMD's Avivo whitepaper says:

"The image output scalers support up to 6 vertical filter taps and up to 10 horizontal filter taps. These scalers are high-precision polyphase scalers that are highly programmable; they are suitable for upscaling by practically any ratio, or for downscaling by up to 4:1."

The question: How can the video scaler hardware be used from a Windows program?

Assume there already exists a decoded video frame, for example in a IDirect3DSurface9, and the goal is to display that video frame on screen using the hardware scaler. I would like to use a Windows API like Media Foundation or DirectShow, rather than vendor-specific APIs if possible. I am mainly interested in upscaling by a fairly large factor around 1.5-3x.

A secondary question is, how can the video scaler hardware parameters be controlled? (For example, the filter coefficients in the polyphase filters mentioned above)

EDIT: Bounty started. Please provide an example of any way to use video scaler hardware in video card (this may be vendor specific, or use any version of DirectX/DirectShow/Media Foundation API).

EDIT: Update: Some examples of programs that do use the video scaler hardware: WinDVD, PowerDVD, madVR. I want to know how to accomplish what they do, which is to use the GPU's builtin video hardware scaler, not a scaler implemented using D3D shaders and texture samplers.

回答1:

To just access hardware - Direct2D (Direct3D is probably a better choice) is fine. But to access the video scaler you'll need to use DirectShow or Windows Media Foundation (I haven't really used this a lot).

AFAIK - the "Resizer DMO" filter will access the correct hardware and perform the video resize for you. Here is a code sample that uses this filter.

EDIT: If you are using MFTs, the Video Processor Filter also ought to be as efficient. As indicated in the documentation, "The video processor supports GPU-accelerated video processing, using Microsoft Direct3D 11. For more information, see MF_SA_D3D11_AWARE."

Also - see this section on hardware MFT attributes, especially MFT_ENUM_HARDWARE_URL_Attribute that lets you check if that MFT is implemented in hardware or not. Note that DXVA integrates with Media Foundation and exposes its functionality as MFTs. So an MFT that is implemented in hardware is most likely using the underlying hardware and is the method used by Windows itself.

Hope this helps!

回答2:

Some of the possible approaches are:

1. Use MFCreateVideoRenderer to create an EVR media sink, and call IMFVideoDisplayControl::SetRenderingPrefs with MFVideoRenderPrefs_AllowScaling flag set (or use IMFAttributes and set the EVRConfig_AllowScaling attribute) and then call IMFVideoDisplayControl::SetVideoPosition to define how the result is scaled. This is part of the Enhanced Video Renderer (EVR).

2. Use IDirectXVideoProcessor::VideoProcessBlt and set DXVA2_VideoProcessBltParams::ConstrictionSize to define how the result is scaled. This is also based on EVR/DXVA.

3. (suggested by ananthonline) Use Video Resizer DSP and use IWMResizerProps::SetFullCropRegion (or MFPKEY_RESIZE_DST_WIDTH and MFPKEY_RESIZE_DST_HEIGHT) to scale the result. This is both a DirectX Media Object (DMO) and Media Foundation Transform (MFT). Note: A video MFT has the attribute MF_SA_D3D_AWARE which can be used to query whether it supports DirectX 3D hardware acceleration, and this can be enabled by sending it the MFT_MESSAGE_SET_D3D_MANAGER message.

4. Use Video Processor MFT and set IMFVideoProcessorControl::SetConstrictionSize to scale the result. This is a MFT.

5. Use a DirectX 3D device and call StretchRect to scale a surface. Note: this pretty obviously does not use the video scaler hardware, it uses texture sampler hardware. A texture can be rendered on a quad with similar effect.

I am still not sure which, if any, of these approaches uses the video scaler hardware. It is likely that at least approaches 1 and 2 would, because they are tied directly to EVR/DXVA; approaches 3 and 4 also might if they are accelerated by DXVA. A definitive answer is still needed, ideally with a reference to documentation and/or a code sample.

回答3:

Direct2D is Microsoft's way of gaining access to the graphics hardware, albeit indirectly. The Direct2D Transform does scaling. You won't have any direct control over the video scaler but must trust the driver to pick the correct settings for you.