UWP speech recognition failure requires restart wi

问题:

I'm trying to use continuous speech recognition in UWP application, but each time after number of successful results in very different random moments during processing ContinuousRecognitionSession_ResultGenerated just stops receive recognition event:

SpeechRecognizer contSpeechRecognizer = new SpeechRecognizer();
        private CoreDispatcher dispatcher;

        protected async override void OnNavigatedTo(NavigationEventArgs e)
        {           
            bool permissionGained = await AudioCapturePermissions.RequestMicrophonePermission();
            if (permissionGained)
            {
                dispatcher = CoreWindow.GetForCurrentThread().Dispatcher;
                contSpeechRecognizer = new Windows.Media.SpeechRecognition.SpeechRecognizer();
                await contSpeechRecognizer.CompileConstraintsAsync();
                contSpeechRecognizer.HypothesisGenerated += ContSpeechRecognizer_HypothesisGenerated;
                contSpeechRecognizer.ContinuousRecognitionSession.ResultGenerated += ContinuousRecognitionSession_ResultGenerated;
                contSpeechRecognizer.ContinuousRecognitionSession.Completed += ContinuousRecognitionSession_Completed;
                await contSpeechRecognizer.ContinuousRecognitionSession.StartAsync();
            }
            else
            {
                this.textBlock1.Text = "Permission to access capture resources was not given by the user, reset the application setting in Settings->Privacy->Microphone.";
            }
        }

        private async void ContinuousRecognitionSession_Completed(SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionCompletedEventArgs args)
        {
            await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                textBlock1.Text = "Timeout.";
            });
            await contSpeechRecognizer.ContinuousRecognitionSession.StartAsync();
        }

        private async void ContSpeechRecognizer_HypothesisGenerated(
            SpeechRecognizer sender, SpeechRecognitionHypothesisGeneratedEventArgs args)
        {
            await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                textBlock3.Text = args.Hypothesis.Text;
            });
        }

        private async void ContinuousRecognitionSession_ResultGenerated(
            SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
        {
            await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                textBlock1.Text = "Waiting ...";
                string r = args.Result.Text.Trim().ToLower();

                TTS(r);               
            });
        }

I found two types of failure behavior during testing:

1. It goes in Timeout with ContinuousRecognitionSession_Completed and remains in this state, in some cases resumes the process. I don't know how to control this condition.

2. Just stops receive args.Result.Text in ContinuousRecognitionSession_ResultGenerated, when unlike the first option ContSpeechRecognizer_HypothesisGenerated continuous receive args.Hypothesis.Text;, then after some time in same session hypothesis event also stops to work

In both cases process interrupted. Same problem described in this question Why does UWP continuous speech recognition stop, but does not have marked or shown answer, as well as in my linked question Send speech recognition args.Result as parameter in UWP desktop-bridge package, but now it is clear that the problem is not connected with Package desktop applications (Desktop Bridge), it is independent problem and appears same way outside the package.

I have tried change it to async and await with private async Task TTS(string r) instead of private async void TTS(string r), as suggested by David Hollowell on WDC [UWP][Desktop Bridge]Send speech recognition args.Result as parameter in UWP desktop-bridge package:

    private async void ContinuousRecognitionSession_ResultGenerated(
        SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionResultGeneratedEventArgs args)
    {
        await dispatcher.RunAsync(CoreDispatcherPriority.Normal, async () =>
        {
            textBlock1.Text = "Waiting ...";
            string r = args.Result.Text.Trim().ToLower();

            await TTS(r);
        });
    }

but result is same, after 1-2 minutes of testing about 30 recognition transaction requests, I got the same failure. It seems like, event disappears before and independently of any used function as is it described in my bridge question with only difference that when accessing the console failure happened immediately after the first request.

I'm trying to figure out, how to restart the ContSpeechRecognizer when it goes into the foreground if it can be a reason of failure, as suggested in answer

回答1:

The current session is interrupted when the software determines that you have not entered it for a long time or the microphone is not of good sound quality (small or unclear).

At this time, if the recognized text already exists in the current session, the ResultGenerated event will be triggered. If you find that your program only triggers the Completed event without triggering the ResultGenerated event, there is no recognized text in the current session.

So if you want to ensure that your voice is recognized correctly, try to ensure the quality of the source. Also, to prevent the ResultGenerated event from firing only once, you can rebind the ResultGenreated event before restarting the session.

private async void ContinuousRecognitionSession_Completed(SpeechContinuousRecognitionSession sender, SpeechContinuousRecognitionCompletedEventArgs args)
{
    await dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
    {
        textBlock1.Text = "Timeout.";
    });
    sender.ResultGenerated -= ContinuousRecognitionSession_ResultGenerated;
    sender.ResultGenerated += ContinuousRecognitionSession_ResultGenerated;
    await sender.StartAsync();
}

Incidentally, there is no direct evidence that rebinding events can ensure that they work properly. But it does improve the success rate of event triggering

Best regards.