React Native. MP3 Binary String (Uint8Array(9549))

Question

I am trying to play an audio file with binary string format that Amazon Polly returns. For that, I am using 'react-native-fetch-blob' and reading a stream, but just keep getting errors from the bridge saying 'Invalid data message - all must be length: 8'. It happens when I try to open the stream: ifstream.open()

This is the code:

//polly config
const params = {
    LexiconNames: [], 
    OutputFormat: "mp3", 
    SampleRate: "8000", 
    Text: "All Gaul is divided into three parts", 
    TextType: "text", 
    VoiceId: "Joanna"
};

Polly.synthesizeSpeech(params, function(err, data) {
    let _data = "";
    RNFetchBlob.fs.readStream(
        // file path
        data.AudioStream,
        // encoding, should be one of `base64`, `utf8`, `ascii`
        'ascii'
    )
    .then((ifstream) => {
        ifstream.open()
        ifstream.onData((chunk) => {
            _data += chunk
        })
        ifstream.onError((err) => {
            console.log('oops', err.toString())
        })
        ifstream.onEnd(() => {  
            //pasing _data to streaming player or normal audio player
            ReactNativeAudioStreaming.play(_data, {showIniOSMediaCenter: true, showInAndroidNotifications: true});
        })  
    })
}); 

Another solution I have also tried is to save the stream into a file to load it later on, but I got similars bugs. RNFetchBlob.fs.createFile("myfile.mp3", dataG.AudioStream, 'ascii');

Huge thanks in advance

Answer 1

You can use fetch() to request one or more media resources, return Response.body.getReader() from .then() to get a ReadableStream of the response. Read the Uint8Array values returned as the stream as read with .read() method of the ReadableStream, append to value to SourceBuffer of MediaSource to stream the media at an HTMLMediaElement.

For example, to output the audio of two requested audio resources, in sequence

window.addEventListener("load", () => {

  const audio = document.createElement("audio");

  audio.controls = "controls";

  document.body.appendChild(audio);

  audio.addEventListener("canplay", e => {
    audio.play();
  });

  const words = ["hello", "world"];

  const mediaSource = new MediaSource();

  const mimeCodec = "audio/mpeg";

  const mediaType = ".mp3";

  const url = "https://ssl.gstatic.com/dictionary/static/sounds/de/0/";

  Promise.all(
      words.map(word =>
        fetch(`https://query.yahooapis.com/v1/public/yql?q=select * from data.uri where url="${url}${word}${mediaType}"&format=json&callback=`)
        .then(response => response.json())
        .then(({
            query: {
              results: {
                url
              }
            }
          }) =>
          fetch(url).then(response => response.body.getReader())
          .then(readers => readers)
        )
      )
    )
    .then(readers => {

      audio.src = URL.createObjectURL(mediaSource);
      mediaSource.addEventListener("sourceopen", sourceOpen);

      async function sourceOpen() {
        var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
        // set `sourceBuffer` `.mode` to `"sequence"`
        sourceBuffer.mode = "segments";

        const processStream = ({
          done,
          value
        }) => {
          if (done) {
            return;
          }
          // append chunk of stream to `sourceBuffer`
          sourceBuffer.appendBuffer(value);
        }
        // at `sourceBuffer` `updateend` call `reader.read()`,
        // to read next chunk of stream, append chunk to 
        // `sourceBuffer`
        for (let [index, reader] of Object.entries(readers)) {
          sourceBuffer.addEventListener("updateend", function() {
            reader.read().then(processStream);
          });

          let stream = await reader.read().then(processStream)
            .then(() => reader.closed)
            .then(() => "done reading stream " + index);

          console.log(stream);
        }

      }
    })
}) 

plnkr http://plnkr.co/edit/9zHwmcdG3UKYMghD0w3q?p=preview

Answer 2

You could use the getSynthesizeSpeechUrl method from AWS.Polly.Presigner. I’m doing this and using react-native-sound to play the mp3. I ran into an issue where the mp3 wouldn’t play because my presigned URL contained special characters, but there’s a fix here.