Streaming voice between Android Phones over WiFi

Question

I'm trying to stream audio from the mic from 1 Android to another over WiFi. After looking at some examples I made 2 applications with a single activity in each, 1 to capture and send audio and the other to receive.

I've used the Audiorecord and Audiotrack classes to capture and play. However, i just hear some crackling sound (which has now stopped after i made some changes though i reverted back)

The activity to send voice.

public class VoiceSenderActivity extends Activity {

private EditText target;
private TextView streamingLabel;
private Button startButton,stopButton;

public byte[] buffer;
public static DatagramSocket socket;
private int port=50005;         //which port??
AudioRecord recorder;

//Audio Configuration. 
private int sampleRate = 8000;      //How much will be ideal?
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       

private boolean status = true;




@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    target = (EditText) findViewById (R.id.target_IP);
    streamingLabel = (TextView) findViewById(R.id.streaming_label);
    startButton = (Button) findViewById (R.id.start_button);
    stopButton = (Button) findViewById (R.id.stop_button);

    streamingLabel.setText("Press Start! to begin");

    startButton.setOnClickListener (startListener);
    stopButton.setOnClickListener (stopListener);
}

private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = false;
                recorder.release();
                Log.d("VS","Recorder released");
    }

};

private final OnClickListener startListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = true;
                startStreaming();           
    }

};

public void startStreaming() {


    Thread streamThread = new Thread(new Runnable() {

        @Override
        public void run() {
            try {


                int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
                DatagramSocket socket = new DatagramSocket();
                Log.d("VS", "Socket Created");

                byte[] buffer = new byte[minBufSize];

                Log.d("VS","Buffer created of size " + minBufSize);
                DatagramPacket packet;

                final InetAddress destination = InetAddress.getByName(target.getText().toString());
                Log.d("VS", "Address retrieved");


                recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,sampleRate,channelConfig,audioFormat,minBufSize);
                Log.d("VS", "Recorder initialized");

                recorder.startRecording();


                while(status == true) {


                    //reading data from MIC into buffer
                    minBufSize = recorder.read(buffer, 0, buffer.length);

                    //putting buffer in the packet
                    packet = new DatagramPacket (buffer,buffer.length,destination,port);

                    socket.send(packet);


                }



            } catch(UnknownHostException e) {
                Log.e("VS", "UnknownHostException");
            } catch (IOException e) {
                Log.e("VS", "IOException");
            } 


        }

    });
    streamThread.start();
 }
 }

The activity to receive voice

public class VoiceReceiverActivity extends Activity {


private Button receiveButton,stopButton;

public static DatagramSocket socket;
private AudioTrack speaker;

//Audio Configuration. 
private int sampleRate = 8000;      //How much will be ideal?
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       

private boolean status = true;


@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    receiveButton = (Button) findViewById (R.id.receive_button);
    stopButton = (Button) findViewById (R.id.stop_button);
    findViewById(R.id.receive_label);

    receiveButton.setOnClickListener(receiveListener);
    stopButton.setOnClickListener(stopListener);

}


private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View v) {
        status = false;
        speaker.release();
        Log.d("VR","Speaker released");

    }

};


private final OnClickListener receiveListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
        status = true;
        startReceiving();

    }

};

public void startReceiving() {

    Thread receiveThread = new Thread (new Runnable() {

        @Override
        public void run() {

            try {

                DatagramSocket socket = new DatagramSocket(50005);
                Log.d("VR", "Socket Created");


                byte[] buffer = new byte[256];


                //minimum buffer size. need to be careful. might cause problems. try setting manually if any problems faced
                int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);

                speaker = new AudioTrack(AudioManager.STREAM_MUSIC,sampleRate,channelConfig,audioFormat,minBufSize,AudioTrack.MODE_STREAM);

                speaker.play();

                while(status == true) {
                    try {


                        DatagramPacket packet = new DatagramPacket(buffer,buffer.length);
                        socket.receive(packet);
                        Log.d("VR", "Packet Received");

                        //reading content from packet
                        buffer=packet.getData();
                        Log.d("VR", "Packet data read into buffer");

                        //sending data to the Audiotrack obj i.e. speaker
                        speaker.write(buffer, 0, minBufSize);
                        Log.d("VR", "Writing buffer content to speaker");

                    } catch(IOException e) {
                        Log.e("VR","IOException");
                    }
                }


            } catch (SocketException e) {
                Log.e("VR", "SocketException");
            }


        }

    });
    receiveThread.start();
}

}

I used wireshark to check if the packets are being sent and i can see the packets. The source however, is the MAC address of the sending device and destination too something like a physical address. Not sure if this is relevant though.

So what's the problem with?

Answer 1

I'd try dividing the problem into Three parts.

Part 1

Ensure the Socket Connection is working fine by commenting everything related to audio

Part 2

Send simply just an arbitrary text message [Hello WiFi] from the sender, then receiving and printing it in the receiver side application.

Part 3

Whether the recorder is actually working? try to test your recording way in separate project to see if it is working properly or not.

Use this code to capture microphone and play it.

My experience

I once worked on a similar project and to test it, what I did was after recording I wrote the recorded audio data as a file on the sdcard

(it would be raw audio, so most music players won't be able to play it... mPlayer should play it I guess)

Answer 2

You need to carefully consider your use of UDP (DatagramSocket class) as your network protocol.

UDP is a lightweight protocol that does not guarantee to maintain the ordering of received packets. This may be part of the reason why the audio is garbled. A packet received out of order will result in a packets worth of audio being played out of order. At the boundary of these out-of-sequence packets you will hear clicks/pops where the audio sample is effectively corrupt. In addition to this, UDP packets aren't guaranteed to be delivered successfully. Any dropped packets will obviously add to any garbling or distortion that is heard.

TCP (Socket class) would be a better option for optimum audio quality. TCP is a more robust protocol that will maintain the order that packets are received. It also has in-built error checking and will resend any dropped packets. However, because of this attentional functionality, TCP has a higher network overhead.

I started this response by saying that you need to carefully consider which protocol you use. This is because there is a case for using either depending on what is important to you..

If you want ultra low latency playback but are happy to sacrifice the audio quality then UDP will work. However, it will take some experimentation to find the best buffer and sample size.

If you want the best possible audio re-production with zero distortion but are happy to introduce slightly more latency then TCP is the route to go.

I can't say how much more latency TCP would add. But it is possible that it could be implemented without impacting the user experience. The only way to find out is to try it and see.

Answer 3

Hey there is an Open Source library called "Libstreaming" that is used for streaming voice/video over the network using WIFI. Just Have a look at it:

https://github.com/fyhertz/libstreaming

There are also some examples provided, kindly have a look at it:

https://github.com/fyhertz/libstreaming-examples

I have used the library to Stream RTSP Audio over the network,hope it may be useful.