How to read frames from a video as bitmaps in UWP

Question

Is it possible to load a video and extract single frames from it (as images) in Universal Windows Applications?

Answer 1

Is it possible to load a video and extract single frames from it (as images) in Universal Windows Applications?

You can use MediaComposition.GetThumbnailAsync to get an image stream from the video. Then you can use RandomAccessStream.CopyAsync to convert the IInputStream to InMemoryRandomAccessStream. We can add the IRandomAccessStream to set BitmapSource.SetSource.

For example:

private async void Button_Click(object sender, RoutedEventArgs e)
{
    FileOpenPicker openPicker = new FileOpenPicker();
    foreach (string extension in FileExtensions.Video)
    {
        openPicker.FileTypeFilter.Add(extension);
    }
    StorageFile file = await openPicker.PickSingleFileAsync();
    var thumbnail = await GetThumbnailAsync(file);
    BitmapImage bitmapImage = new BitmapImage();
    InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
    await RandomAccessStream.CopyAsync(thumbnail, randomAccessStream);
    randomAccessStream.Seek(0);
    bitmapImage.SetSource(randomAccessStream);
    MyImage.Source = bitmapImage;
}

public async Task<IInputStream> GetThumbnailAsync(StorageFile file)
{
    var mediaClip = await MediaClip.CreateFromFileAsync(file);
    var mediaComposition = new MediaComposition();
    mediaComposition.Clips.Add(mediaClip);
    return await mediaComposition.GetThumbnailAsync(
        TimeSpan.FromMilliseconds(5000), 0, 0, VideoFramePrecision.NearestFrame);
}

internal class FileExtensions
{
    public static readonly string[] Video = new string[] { ".mp4", ".wmv" };
}

Answer 2

There are two namespaces that allow you to get frames:

1. "Windows.Media.Capture" namespace. Use it if you want to capture video from camera then read "Process media frames with MediaFrameReader" https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/process-media-frames-with-mediaframereader
2. "Windows.Media.Playback" namespace. Use it if you want to get frames from file or stream video. Scroll to paragraph "Use MediaPlayer in frame server mode" on https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/play-audio-and-video-with-mediaplayer