Lazy loading of non-visible elements

Question

I have a case where I have either a gridview/listbox/any type of items control and the number of items bound to the control is massive (easily around 5000+ mark).

Each of these items needs to have various attributes loaded from various web services. Obviously, reaching out to web services to process this amount of elements all at once is out of the question.

My question is, is it possible to postpone loading until these items are actually displayed to the user? As in, the user scrolls down and although the items have been present in the collection all along, they are processed only when they are actually physically rendered.

I've seen it done before, but I can't remember where exactly. It was a situation where lots of stock quotes were in a collection bound to a gridview, but their attributes (prices etc...) were empty until they were displayed for the first time (by scrolling to their respective position).

Hopefully this made (some) sense.

Any ideas on how to pull it off?

Answer 1

I would try a combination of lazy loading and asynchronous loading:
Use a virtualizing list-control. Create a ViewModel for your items and fill your list with instances of the ViewModel (one per line).

In your ViewModel, make properties that have a default-value that shows the user that the data not has been loaded. The first time one of these property is accessed, trigger loading the data asynchronous and fire INotifyPropertyChanged when the real data has been received.

This will give the user a nice experience and most of the tricky work will be done through the virtualizing list (in WPF this are ListBox,ListView, DataGrid...). Hope this helped.

class LineItemVM : INotifyPropertyChanged{

  bool   m_loadingTriggered;
  string m_name="Loading...";
  string m_anotherProperty="Loading...";


  public string Name{
     get{
       TriggerLoadIfNecessary(); // Checks if data must be loaded
       return m_name;
     }
  }

  public string AnotherProperty{
     get{
       TriggerLoadIfNecessary(); // Checks if data must be loaded
       return m_anotherProperty;
     }
  }


  void TriggerLoadIfNecessary(){        
     if(!m_loadingTriggered){
       m_loadingTriggered=true;

       // This block will called before your item will be displayed
       //  Due to the m_loadingTriggered-member it is called only once.
       // Start here the asynchronous loading of the data
       // In virtualizing lists, this block is only called if the item
       //  will be visible to the user (he scrolls to this item)

       LoadAsync();
     }
  }

  ...

Additional logic As an idea, you could also make an outer asynchrounous loading thread that loads all data in background, but has a list for items that should be loaded with higher priority. The concept is the same as in the above example, but instead of loading data from your ViewModel-item, the TriggerLoadIfNecessary-method only adds the item in the high-priority list so that the potentially visible elements are loaded first. The question which version is better suited depends on the usage of the list. If it is probable that the user uses the full list and does not navigate quickly away, this extended version is better. Otherwise the original version is probably better.

Answer 2

Here is an event that will notify when user scrolls into the last screen of data:

using System.Windows;
using System.Windows.Controls;

public static class ScrollViewer
{
    public static readonly RoutedEvent LastPageEvent = EventManager.RegisterRoutedEvent(
        "LastPage",
        RoutingStrategy.Bubble,
        typeof(RoutedEventHandler),
        typeof(ScrollViewer));

    private static readonly RoutedEventArgs EventArgs = new RoutedEventArgs(LastPageEvent);

    static ScrollViewer()
    {
        EventManager.RegisterClassHandler(
            typeof(System.Windows.Controls.ScrollViewer),
            System.Windows.Controls.ScrollViewer.ScrollChangedEvent,
            new ScrollChangedEventHandler(OnScrollChanged));
    }
    public static void AddLastPageHandler(UIElement e, RoutedEventHandler handler)
    {
        e.AddHandler(LastPageEvent, handler);
    }

    public static void RemoveLastPageHandler(UIElement e, RoutedEventHandler handler)
    {
        e.RemoveHandler(LastPageEvent, handler);
    }

    private static void OnScrollChanged(object sender, ScrollChangedEventArgs e)
    {
        if (e.ViewportHeight == 0 || e.VerticalOffset == 0)
        {
            return;
        }

        var verticalSpaceLeft = e.ExtentHeight - e.VerticalOffset;
        if (verticalSpaceLeft < 2 * e.ViewportHeight)
        {
            var scrollViewer = (System.Windows.Controls.ScrollViewer)sender;
            scrollViewer.RaiseEvent(EventArgs);
        }
    }
}