How to fix notifyDataSetChanged/ListView problems

Summary: Trying to dynamically add heading rows to a ListView via a custom adapter wrapper. ListView is having trouble keeping the scroll position in sync. Runnable demo project provided.

I would like to dynamically add items to a list based on the values in a CursorAdapter, several positions ahead of what the user is currently viewing. To do this, I have an adapter that wraps the CursorAdapter and keeps the new content indexed in a SparseArray. The ListView needs to be updated when items are added to the custom adapter, but I have met a lot of pitfalls trying to get that to work and would love some advice.

The demo project can be downloaded here: DynamicSectionedList.zip

In the demo, the headings are added dynamically by looking ahead 10 places to find the correct position where the list items switch to the next letter. Each implementation of notifyDataSetChanged has problems as described:

Demo 1 This demo shows the importance of notifyDataSetChanged(). On clicking anything, the app will crash. This is due to some sanity checking in ListView... mItemCount != adapter.getItemCount(). Moral is, we need to notify the list that data has changed.

Demo 2 The natural next step is to notify the ListView of changes when changes occur. Unfortunately, doing so while the ListView is scrolling firmly breaks all touch interaction until the app switches out of touch mode. You will need to "fling scroll" far enough to generate new headings in order to notice this. Tapping the screen will not cause the scroll to stop, and once stopped none of the list items will be clickable. This is due to some if (!mDataChanged) { /* do very important stuff */ } code in AbsListView.onTouchEvent().

Demo 3 To fix this, Demo 3 introduces a pendingChanges flag and the custom Adapter gains a notifyDataSetChangedIfNeeded() which can be called by the ListView once it has entered a "safe" state for changes. The first point where changes must be notified is in ListView.layoutChildren(), so I overrode that method to first notify of changes if needed, then call through. Fling past at least one heading then click a list item.

This doesn't quite work right, though I'm not totally sure why. Tapping or selecting an item with the keyboard/trackball causes the list to refresh without properly syncing the old position. It scrolls to the top of the list which is not acceptable.

Demo 4 The scroll problem in Demo 3 can be conquered, at least in touch mode. By adding a call to notifyDataSetChangedIfNeeded() on touch down, the data change happens to take place at such a time that all touch interaction works as expected and the list position is properly synced.

However, I can't find an analog for that when the device is not in touch mode, not to mention the fact that it definitely seems like a hack. The list almost always scrolls back to the top, I can't find out what causes it to occasionally maintain the correct position.

Since Android is fighting me at each step of the way, I feel like there should be a better approach. Please try the demo, if any fixes can be applied to get it working that would be great!

Many thanks to anyone who can look into this, hopefully if we can get the code working it will be useful for others trying to accomplish the same optimization for lists with headings.

The main problem with the approach presented above was that the data set was being changed directly from within code executed by the superclasses. By populating the headings in getView() I was changing the data in what could be the middle of an operation or an "unsafe" state in the superclasses. This is why all touch interaction broke when onNotifyDataSetChanged was called during a fling scroll. The data needs to be added from an external source, not from within the adapter methods.

That can be done by using a Handler or AsyncTask to add the headings to the adapter and call its notifyDataSetChanged. These will be run on a UI thread and will not interfere with the superclass state. Thanks to CommonsWare's EndlessAdapter example for introducing the concept of an AsyncTask to add data to a list.

The onTouchEvent() and layoutChildren() hacks were no longer needed after making that change.