Windows Phone 8.1 - Data virtualization with pictu

Question

I'm building a Windows Phone 8.1 app which requires me to display all images in the pictures library in a GridView. I have built a class named VirtualList which is a list that supports IncrementalLoading, and I have added all the images from pictures library to that list. When there are a reduced number of images (less than 80 photos), everything works fine, but when there are more than 80 photos, the app shuts down due to an OutOfMemoryException. I suppose that the items that aren't displayed at the moment are not kept in memory, or are they? For my purpose, should I continue using incremental loading, or should I switch to random access data virtualization? If I should switch to random access data virtualization, could you provide me with an example about how to implement that?

My code is as follows:

VirtualList.cs

class VirtualList : List<Windows.UI.Xaml.Media.ImageSource>, ISupportIncrementalLoading
{

    private IReadOnlyList<StorageFile> photos;

    public VirtualList(IReadOnlyList<StorageFile> files) : base()
    {
        photos = files;
    }

    public bool HasMoreItems
    {
        get 
        {
            return this.Count < photos.Count;
        }
    }

    public Windows.Foundation.IAsyncOperation<LoadMoreItemsResult> LoadMoreItemsAsync(uint count)
    {
        return LoadMoreItemsAwaitable(count).AsAsyncOperation<LoadMoreItemsResult>();
    }

    private async Task<LoadMoreItemsResult> LoadMoreItemsAwaitable(uint count)
    {
        for (int i = Count; i < Count + count; i++)
        {
            using (var fileStream = await photos[i].OpenAsync(FileAccessMode.Read))
            {
                BitmapImage bitmapImage = new BitmapImage();
                await bitmapImage.SetSourceAsync(fileStream);
                this.Add(bitmapImage);
            }
        }

        return new LoadMoreItemsResult { Count = count };
    }
}

XAML code (MainPage.xaml):

<GridView x:Name="photosGrid" Height="392" Width="400" ItemsSource="{Binding}" Margin="0,0,-0.333,0" SelectionMode="Multiple" Background="Black">
                    <GridView.ItemTemplate>
                        <DataTemplate>   
                            <Image Width="90" Height="90" Margin="5" Source="{Binding}" Stretch="UniformToFill"/>
                        </DataTemplate>
                    </GridView.ItemTemplate>

                </GridView>

MainPage.xaml.cs code

//This code is inside OnNavigatedTo method
var files = await KnownFolders.CameraRoll.GetFilesAsync();
VirtualList imageList = new VirtualList(files);
photosGrid.DataContext = imageList;

Answer 1

In addition to Kevin's feedback, I recommend using GetThumbnailAsync() instead of pulling the full size image into memory.

Another thing you can consider is using a Converter so that the image thumbnail only gets loaded when the binding is called, then you can use Deferred Rendering on the ItemTemplate (Deferred Rendering is a great way to improve the performance of large/long lists)

Here is one of my ThumbnailConverters:

public class ThumbnailToBitmapImageConverter : IValueConverter
{
    public object Convert(object value, Type targetType, object parameter, string language)
    {
        if (DesignMode.DesignModeEnabled)
            return "images/VideoFileIcon.png";

        var fileName = (string) value;

        if (string.IsNullOrEmpty(fileName))
            return "";

        return GetThumbnailImage(fileName);
    }

    private async Task<BitmapImage> GetThumbnailImage(string fileName)
    {
        try
        {
            var file = await ApplicationData.Current.LocalFolder.GetFileAsync(fileName)
                .AsTask().ConfigureAwait(false);
            var thumbnail = await file.GetScaledImageAsThumbnailAsync(ThumbnailMode.ListView, 90, ThumbnailOptions.UseCurrentScale)
                .AsTask().ConfigureAwait(false);
            var bmi = new BitmapImage();
            bmi.SetSource(thumbnail);
            return bmi;
        }
        catch (Exception ex)
        {
           await new MessageDialog(ex.Message).ShowAsync();
        }
        return null;
    }

    public object ConvertBack(object value, Type targetType, object parameter, string language)
    {
        throw new NotImplementedException();
    }
}

Answer 2

Many problems here.

The first one is that your collection needs to implement INotifyPropertyChanged to properly support incremental loading (don't ask me why). Fortunately it's easy to fix: just inherit from ObservableCollection<T> instead of List<T>.

The second issue comes from your implementation of LoadMoreItemsAwaitable. More specifically, the for loop:

for (int i = Count; i < Count + count; i++)
{
    using (var fileStream = await photos[i].OpenAsync(FileAccessMode.Read))
    {
        BitmapImage bitmapImage = new BitmapImage();
        await bitmapImage.SetSourceAsync(fileStream);
        this.Add(bitmapImage);
    }
}

Every time you add an item to the collection (this.Add(bitmapImage)), the value of Count increases. The result is that i and Count both increase at the same time, making your loop infinite. To prevent that, save the value of Count outside of the loop:

int offset = this.Count;

for (int i = offset; i < offset + count && i < photos.Count; i++)
{
    using (var fileStream = await photos[i].OpenAsync(FileAccessMode.Read))
    {
        BitmapImage bitmapImage = new BitmapImage();
        await bitmapImage.SetSourceAsync(fileStream);
        this.Add(bitmapImage);
    }
}

Note that I also check that i is lower than photos.Count, otherwise you could have an ArgumentOufOfRangeException.

From this point, you can try and you'll see that it'll work. Still, if you scroll down your list, the memory will perpetually increase. Why? Because you're storing your BitmapImage, thus nullifying the benefits of virtualization.

Let me explain:

1. At first, your grid displays, say, five elements. so five elements are loaded in your collection.
2. The user scrolls down, the grid needs to display the next five elements. It loads five new elements from your collection, and throws away the five previous ones to save memory (thanks to virtualization). Problem: you still store those five items in your collection! Therefore, the memory is never freed.

Unfortunately, I don't think there's a perfect way to solve this with ISupportIncrementalLoading (there's no API to tell that the grid needs to re-display previous elements, so you need to keep them at all time). But you can still avoid hogging the memory by storing just the path of the file instead of the BitmapImage.

Problem: there is a way to populate an Image control by providing just the path (by using the ms-appx URI scheme), but as far as I know it doesn't work for pictures stored in the Picture Library. So you do need to return BitmapImage controls at some point. At first, I thought about writing a converter (that would convert the path to BitmapImage, but it requires asynchronous APIs, and converter are synchronous... The easiest solution I could think was to make your own Image control, that can load this kind of path. The Image control is sealed so you can't directly inherit from it (sometimes, I think WinRT has been specifically designed to annoy developers), but you can wrap it in a UserControl.

Let's create a UserControl called LocalImage. The XAML just wraps the Image control:

<UserControl
    x:Class="StackOverflowUniversal10.LocalImage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="300"
    d:DesignWidth="400">

    <Image x:Name="Image" Width="90" Height="90" Margin="5" Source="{Binding}" Stretch="UniformToFill"/>
</UserControl>

In the code-behind, we create a dependency property, and use it to load the picture:

public sealed partial class LocalImage : UserControl
{
    public static readonly DependencyProperty SourceProperty = DependencyProperty.Register("Source", typeof (string),
        typeof (LocalImage), new PropertyMetadata(null, SourceChanged));

    public LocalImage()
    {
        this.InitializeComponent();
    }

    public string Source
    {
        get { return this.GetValue(SourceProperty) as string; }
        set { this.SetValue(SourceProperty, value); }
    }

    private async static void SourceChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        var control = (LocalImage)obj;

        var path = e.NewValue as string;

        if (string.IsNullOrEmpty(path))
        {
            control.Image.Source = null;
        }
        else
        {
            var file = await StorageFile.GetFileFromPathAsync(path);

            using (var fileStream = await file.OpenAsync(FileAccessMode.Read))
            {
                BitmapImage bitmapImage = new BitmapImage();
                await bitmapImage.SetSourceAsync(fileStream);
                control.Image.Source = bitmapImage;
            }
        }
    }
}

Then modify your page to use the UserControl instead of the Image control:

<GridView x:Name="photosGrid" Height="382" Width="400" ItemsSource="{Binding}" Margin="0,0,-0.333,0" SelectionMode="Multiple" Background="Black">
    <GridView.ItemTemplate>
        <DataTemplate>
            <local:LocalImage Width="90" Height="90" Margin="5" Source="{Binding}"/>
        </DataTemplate>
    </GridView.ItemTemplate>
</GridView>

Last but not least, change your collection to store the paths instead of the pictures:

class VirtualList : ObservableCollection<string>, ISupportIncrementalLoading
{
    private IReadOnlyList<StorageFile> photos;

    public VirtualList(IReadOnlyList<StorageFile> files) : base()
    {
        photos = files;
    }

    public bool HasMoreItems
    {
        get
        {
            return this.Count < photos.Count;
        }
    }

    public Windows.Foundation.IAsyncOperation<LoadMoreItemsResult> LoadMoreItemsAsync(uint count)
    {
        return LoadMoreItemsAwaitable(count).AsAsyncOperation<LoadMoreItemsResult>();
    }

    private async Task<LoadMoreItemsResult> LoadMoreItemsAwaitable(uint count)
    {
        int offset = this.Count;

        for (int i = offset; i < offset + count && i < photos.Count; i++)
        {
            this.Add(photos[i].Path);
        }

        return new LoadMoreItemsResult { Count = count };
    }
}

And it should work, with stable memory consumption. Note that you can (and should) further reduce the memory consumption by setting the DecodePixelHeight and DecodePixelWidth properties of your BitmapImage (so that the runtime will load a thumbnail in memory rather than the full-res picture).