I'm trying to build a Google Assistant app to identify an audio clip that's playing from another device (TV, Radio, etc..) and take an action based on that Audio clip to play another audio clip, YouTube video or open a website in a browser.
It will be great to know the possibility of implementing this feature and if there is any available Google resource/ Article that I can use.
Thanks,
I'm sorry to inform you that as of now, you do not get access to the raw recordings of interactions with the Assistant. You only get the string of user's utterance. This means that you can't develop an action that identifies audio clips, you can only use strings in them (assuming that speech-to-text engine can understand them)
You also can't open a website or YouTube app based on ANY input, even if you use strings. Because the interactions with the Assistant are voice-first. Meaning that you can't always assume that the user has a screen or has the YouTube app installed. What if the interaction is taking place through an Assistant-enabled smart speaker?
You can however, play user's voice back to them (or someone else's voice) by recording it in a separate app and then using SSML syntax or Media Responses of the Assistant to play them. Here's a detailed article of what I mean and it can be implemented.
