WPF touch application (partially) freezes on .NET

Question

Update

Microsoft acknowledged the issue:

Gepost door Microsoft op 13/10/2017 om 11:38

Thank you for reporting this. We are aware of this issue and are fixing it in a future version of .NET. There is also a related issue that is being released in a servicing fix that will drastically reduce the possibility of hitting this problem. This will be serviced relatively soon.

Problem

Our WPF application is being used on tablets using touch (no stylus) and we are experiencing issues after the installation of .NET Framework 4.7. Two scenarios can occur after using the application for a while: either the application freezes completely and has to be restarted or all touch functionality in Popup or Window elements is disabled. There is quite a difference between the two but I believe the cause is the same.

Scenario 1: full freeze

• The application becomes fully unresponsive, the application must be closed using the Task Manager
• Touch nor mouse can be used
• Sometimes the following error is thrown before the application hangs:

Index was outside the bounds of the array.

This is the stacktrace:

   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Windows.Input.StylusWisp.WispLogic.CoalesceAndQueueStylusEvent(RawStylusInputReport inputReport)
   at System.Windows.Input.StylusWisp.WispLogic.ProcessSystemEvent(PenContext penContext, Int32 tabletDeviceId, Int32 stylusDeviceId, Int32 timestamp, SystemGesture systemGesture, Int32 gestureX, Int32 gestureY, Int32 buttonState, PresentationSource inputSource)
   at System.Windows.Input.PenContext.FireSystemGesture(Int32 stylusPointerId, Int32 timestamp)
   at System.Windows.Input.PenThreadWorker.FireEvent(PenContext penContext, Int32 evt, Int32 stylusPointerId, Int32 cPackets, Int32 cbPacket, IntPtr pPackets)
   at System.Windows.Input.PenThreadWorker.ThreadProc()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart() 

Scenario 2: partial freeze

• The main window is still responsive (by mouse and touch) but any 'overlay' content (Modal Dialog, Window, Popup element from DatePicker, ComboBox, ...) doesn't respond to tapping. The application must be restarted to reenable touch.
• Mouse can still be used in 'overlay' elements.

This issue is also explained in detail here. A video of the behavior after the issue occurs can be found here.

Additional Info

• Both scenarios can be simulated on different types of tablets and also on the Windows Simulator, using a mix of Windows 8.1 and Windows 10.
• The issues are fixed when removing .NET Framework 4.7
• Scenario 2 can be easily reproduced by quickly tapping some ComboBox elements with multiple fingers. After a few minutes, the popup is no longer responsive to touch.
• Scenario 1 is harder to simulate and occurs randomly.

Cause

The issue seems to have something to do with the StylusWisp code. I guess it suddenly fails and becomes unusable after that point.

When disabling the Stylus support by either using DisableWPFTabletSupport or DisableStylusAndTouchSupport, the issue disappears. However, any ScrollViewer with PanningMode="Both" can't be swipe scrolled anymore.

Solution?

A similar issue has been reported to Microsoft. Since there is not much support yet, a fix might take a while. In the meantime I'm looking for a solution for this issue that doesn't involve disabling the .NET Framework 4.7 and that keeps the original touch support intact. Does anyone have the same issues and a better solution?

Answer 1

Update: Below workaround for a workaround solution does not work well.
The problem is that all finger taps are interpreted as mouse clicks. The custom touch scroll only works smoothly for content which does not react on mouse clicks. To make it working fine you would need to find a way to "eat" the mouse click events when a scroll action is performed.

I may have found a workaround for the broken touch scroll.
Handle WM_TOUCH and use a custom TouchDevice.
Credit goes to Luca Cornazzani: Enable multitouch on WPF controls
Another source i used (for the TOUCHINPUT definition) : WPF and multi-touch

On app startup call the well-known DisableWPFTabletSupport function.

MainWindow.xaml:

<Window x:Class="MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:TouchScrollTest"
        mc:Ignorable="d"
        Title="MainWindow" Height="395.603" Width="525">
    <Grid>
        <StackPanel>
            <ComboBox x:Name="comboBox1" FontSize="16" Width="150">
            </ComboBox>

            <ScrollViewer Height="300" Width="300" PanningMode="Both" HorizontalScrollBarVisibility="Auto" VerticalScrollBarVisibility="Auto">
                <TextBlock x:Name="textBlock1">
                </TextBlock>
            </ScrollViewer>
        </StackPanel>        
    </Grid>
</Window>

MainWindow.xaml.vb:

Class MainWindow

    Private _devices As New Dictionary(Of Integer, TouchDeviceEmulator)()

    Public Sub New()

        ' This call is required by the designer.
        InitializeComponent()

        ' Add any initialization after the InitializeComponent() call.

        For i As Integer = 1 To 19
            Dim myComboBoxItem As ComboBoxItem = New ComboBoxItem
            myComboBoxItem.Content = "ComboBoxItem " & i.ToString()
            comboBox1.Items.Add(myComboBoxItem)

        Next

        For i As Integer = 65 To 90
            Dim c As Char = ChrW(i)
            For j As Integer = 1 To 10
                textBlock1.Text += " ".PadLeft(10, c)
            Next
            textBlock1.Text += vbCrLf
        Next

    End Sub

    Protected Overrides Sub OnSourceInitialized(e As EventArgs)
        MyBase.OnSourceInitialized(e)

        Dim source As Interop.HwndSource = TryCast(PresentationSource.FromVisual(Me), Interop.HwndSource)
        source.AddHook(New Interop.HwndSourceHook(AddressOf WndProc))

        Dim presentation = DirectCast(PresentationSource.FromDependencyObject(Me), Interop.HwndSource)
        If presentation Is Nothing Then
            Throw New Exception("Unable to find the parent element host.")
        End If

        RegisterTouchWindow(presentation.Handle, TouchWindowFlag.WantPalm)
    End Sub

    Private Function WndProc(hwnd As IntPtr, msg As Integer, wParam As IntPtr, lParam As IntPtr, ByRef handled As Boolean) As IntPtr

        ' Handle messages...
        If msg = WM_TOUCH Then
            handled = HandleTouch(wParam, lParam)
            Return New IntPtr(1)
        End If

        Return IntPtr.Zero

    End Function

    Private Function HandleTouch(wParam As IntPtr, lParam As IntPtr) As Boolean
        Dim handled As Boolean = False
        Dim inputCount = wParam.ToInt32() And &HFFFF
        Dim inputs = New TOUCHINPUT(inputCount - 1) {}

        If GetTouchInputInfo(lParam, inputCount, inputs, Runtime.InteropServices.Marshal.SizeOf(inputs(0))) Then

            For i As Integer = 0 To inputCount - 1
                Dim input As TOUCHINPUT = inputs(i)
                'TOUCHINFO point coordinates and contact size is in 1/100 of a pixel; convert it to pixels.
                'Also convert screen to client coordinates.
                Dim position As Point = PointFromScreen(New System.Windows.Point((input.x * 0.01), (input.y * 0.01)))

                Dim device As TouchDeviceEmulator = Nothing
                If Not _devices.TryGetValue(input.dwID, device) Then
                    device = New TouchDeviceEmulator(input.dwID)
                    _devices.Add(input.dwID, device)
                End If

                device.Position = position

                If (input.dwFlags And TOUCHEVENTF_DOWN) > 0 Then
                    device.SetActiveSource(PresentationSource.FromVisual(Me))
                    device.Activate()
                    device.ReportDown()
                ElseIf device.IsActive AndAlso (input.dwFlags And TOUCHEVENTF_UP) > 0 Then
                    device.ReportUp()
                    device.Deactivate()
                    _devices.Remove(input.dwID)
                ElseIf device.IsActive AndAlso (input.dwFlags And TOUCHEVENTF_MOVE) > 0 Then
                    device.ReportMove()
                End If
            Next

            CloseTouchInputHandle(lParam)
            handled = True
        End If

        Return handled
    End Function



    Private Class TouchDeviceEmulator
        Inherits TouchDevice

        Public Position As System.Windows.Point

        Public Sub New(deviceId As Integer)
            MyBase.New(deviceId)
        End Sub

        Public Overrides Function GetTouchPoint(relativeTo As IInputElement) As TouchPoint
            Dim pt As System.Windows.Point = Position
            If relativeTo IsNot Nothing Then
                pt = ActiveSource.RootVisual.TransformToDescendant(DirectCast(relativeTo, Visual)).Transform(Position)
            End If

            Dim rect = New Rect(pt, New Size(1.0, 1.0))
            Return New TouchPoint(Me, pt, rect, TouchAction.Move)
        End Function

        Public Overrides Function GetIntermediateTouchPoints(relativeTo As IInputElement) As TouchPointCollection
            Throw New NotImplementedException()
        End Function

        Public Overloads Sub SetActiveSource(activeSource As PresentationSource)
            MyBase.SetActiveSource(activeSource)
        End Sub

        Public Overloads Sub Activate()
            MyBase.Activate()
        End Sub

        Public Overloads Sub ReportUp()
            MyBase.ReportUp()
        End Sub

        Public Overloads Sub ReportDown()
            MyBase.ReportDown()
        End Sub

        Public Overloads Sub ReportMove()
            MyBase.ReportMove()
        End Sub

        Public Overloads Sub Deactivate()
            MyBase.Deactivate()
        End Sub

    End Class



    Private Const WM_TOUCH As Integer = &H240

    Private Enum TouchWindowFlag As UInteger
        FineTouch = &H1
        WantPalm = &H2
    End Enum

    ' Touch event flags ((TOUCHINPUT.dwFlags) [winuser.h]
    Private Const TOUCHEVENTF_MOVE As Integer = &H1
    Private Const TOUCHEVENTF_DOWN As Integer = &H2
    Private Const TOUCHEVENTF_UP As Integer = &H4
    Private Const TOUCHEVENTF_INRANGE As Integer = &H8
    Private Const TOUCHEVENTF_PRIMARY As Integer = &H10
    Private Const TOUCHEVENTF_NOCOALESCE As Integer = &H20
    Private Const TOUCHEVENTF_PEN As Integer = &H40
    Private Const TOUCHEVENTF_PALM As Integer = &H80

    <Runtime.InteropServices.StructLayout(Runtime.InteropServices.LayoutKind.Sequential)>
    Private Structure TOUCHINPUT
        Public x As Int32
        Public y As Int32
        Public hSource As IntPtr
        Public dwID As Int32
        Public dwFlags As Int32
        Public dwMask As Int32
        Public dwTime As Int32
        Public dwExtraInfo As IntPtr
        Public cxContact As Int32
        Public cyContact As Int32
    End Structure

    <Runtime.InteropServices.DllImport("user32")>
    Private Shared Function RegisterTouchWindow(hWnd As System.IntPtr, flags As TouchWindowFlag) As Boolean
    End Function

    <Runtime.InteropServices.DllImport("user32")>
    Private Shared Function GetTouchInputInfo(hTouchInput As IntPtr, cInputs As Int32, <Runtime.InteropServices.[In], Runtime.InteropServices.Out> pInputs As TOUCHINPUT(), cbSize As Int32) As <Runtime.InteropServices.MarshalAs(Runtime.InteropServices.UnmanagedType.Bool)> [Boolean]
    End Function

    <Runtime.InteropServices.DllImport("user32")>
    Private Shared Sub CloseTouchInputHandle(lParam As System.IntPtr)
    End Sub

End Class

Most code in this sample is identical to Cornazzani's C# code.
At least for the ScrollViewer it seems to work, have not tested other controls.
It does not solve my problem with the partially broken Stylus support. Writing on InkCanvas works not as smoothly as before and eraser button does not work at all with the DisableWPFTabletSupport hack.

Also interesting, same approach: WmTouchDevice on github.

Answer 2

Installing the .NET Framework 4.7.1 seems to fix the issue. The .NET Framework 4.7.1 is also included in the Windows 10 Fall Creators Update which has started rolling out since October.