Android app to record sound in real time and ident

Question

I need to develop an app to record frequencies in real time using the phone's mic and then display them (in text). I am posting my code here. The FFT and complex classes have been used from http://introcs.cs.princeton.edu/java/97data/FFT.java.html and http://introcs.cs.princeton.edu/java/97data/Complex.java.html .The problem is when i run this on the emulator the frequency starts from some random value and keeps on increasing till 7996. It then repeats the whole process. Can someone plz help me out?

public class Main extends Activity {

TextView disp;
private static int[] sampleRate = new int[] { 44100, 22050, 11025, 8000 };
short audioData[];
double finalData[];
int bufferSize,srate;
String TAG;
public boolean recording;
AudioRecord recorder;
Complex[] fftArray;
float freq;


@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    disp = (TextView) findViewById(R.id.display);

    Thread t1 = new Thread(new Runnable(){

        public void run() {

            Log.i(TAG,"Setting up recording");
            for (int rate : sampleRate) {
                try{

                    Log.d(TAG, "Attempting rate " + rate);

            bufferSize=AudioRecord.getMinBufferSize(rate,AudioFormat.CHANNEL_CONFIGURATION_MONO,
            AudioFormat.ENCODING_PCM_16BIT)*3; //get the buffer size to use with this audio record

            if (bufferSize != AudioRecord.ERROR_BAD_VALUE) {

            recorder = new AudioRecord (MediaRecorder.AudioSource.MIC,rate,AudioFormat.CHANNEL_CONFIGURATION_MONO,
            AudioFormat.ENCODING_PCM_16BIT,2048); //instantiate the AudioRecorder
            Log.d(TAG, "BufferSize " +bufferSize);
            srate = rate;

            }

            } catch (Exception e) {
                Log.e(TAG, rate + "Exception, keep trying.",e);
            }
        }
            bufferSize=2048;
            recording=true; //variable to use start or stop recording
            audioData = new short [bufferSize]; //short array that pcm data is put into.
            Log.i(TAG,"Got buffer size =" + bufferSize);                
            while (recording) {  //loop while recording is needed
                   Log.i(TAG,"in while 1");
            if (recorder.getState()==android.media.AudioRecord.STATE_INITIALIZED) // check to see if the recorder has initialized yet.
            if (recorder.getRecordingState()==android.media.AudioRecord.RECORDSTATE_STOPPED)
            recorder.startRecording();  //check to see if the Recorder has stopped or is not recording, and make it record.

            else {
                   Log.i(TAG,"in else");
                  // audiorecord();
                finalData=convert_to_double(audioData);
                Findfft();
                for(int k=0;k<fftArray.length;k++)
                {
                    freq = ((float)srate/(float) fftArray.length) *(float)k;
                    runOnUiThread(new Runnable(){
                     public void run()
                     {
                         disp.setText("The frequency is " + freq);
                         if(freq>=15000)
                             recording = false;
                     }
                 });


                }


             }//else recorder started

    } //while recording

    if (recorder.getState()==android.media.AudioRecord.RECORDSTATE_RECORDING) 
    recorder.stop(); //stop the recorder before ending the thread
    recorder.release(); //release the recorders resources
    recorder=null; //set the recorder to be garbage collected.

         }//run

    });
    t1.start();
}





private void Findfft() {
    // TODO Auto-generated method stub
    Complex[] fftTempArray = new Complex[bufferSize];
    for (int i=0; i<bufferSize; i++)
    {
        fftTempArray[i] = new Complex(finalData[i], 0);
    }
    fftArray = FFT.fft(fftTempArray);
}


private double[] convert_to_double(short data[]) {
    // TODO Auto-generated method stub
    double[] transformed = new double[data.length];

    for (int j=0;j<data.length;j++) {
    transformed[j] = (double)data[j];
    }

    return transformed;

}


@Override
public boolean onCreateOptionsMenu(Menu menu) {
    // Inflate the menu; this adds items to the action bar if it is present.
    getMenuInflater().inflate(R.menu.main, menu);
    return true;
 }
 }       

Answer 1

Your question has been succinctly answered, however, to further your objectives and complete the loop...

Yes, FFT is not optimal on limited CPUs for pitch / frequency identification. A more optimal approach is YIN described here. You may find an implementation at Tarsos. Issues you will face are the lack of javax.sound.sampled in the ADK and therefore converting the shorts/bytes from AudioRecord to the floats required for the referenced implementations.

Answer 2

i find this solution after few days - the Best for getting frequency in Hrz:

download Jtransforms and this Jar also - Jtransforms need it.

then i use this task:

public class MyRecorder extends AsyncTask<Void, short[], Void> {

int blockSize = 2048;// = 256;
private static final int RECORDER_SAMPLERATE = 8000;
private static final int RECORDER_CHANNELS = AudioFormat.CHANNEL_IN_MONO;
private static final int RECORDER_AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT;
int BufferElements2Rec = 1024; // want to play 2048 (2K) since 2 bytes we use only 1024
int BytesPerElement = 2;



@Override
protected Void doInBackground(Void... params) {

    try {
        final AudioRecord audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
                RECORDER_SAMPLERATE, RECORDER_CHANNELS,
                RECORDER_AUDIO_ENCODING, BufferElements2Rec * BytesPerElement);
        if (audioRecord == null) {
            return null;
        }

        final short[] buffer = new short[blockSize];
        final double[] toTransform = new double[blockSize];
        audioRecord.startRecording();
        while (started) {
            Thread.sleep(100);
            final int bufferReadResult = audioRecord.read(buffer, 0, blockSize);
            publishProgress(buffer);
        }
        audioRecord.stop();
        audioRecord.release();
    } catch (Throwable t) {
        Log.e("AudioRecord", "Recording Failed");
    }
    return null;
}



@Override
protected void onProgressUpdate(short[]... buffer) {
    super.onProgressUpdate(buffer);
    float freq = calculate(RECORDER_SAMPLERATE, buffer[0]);
    }


public static float calculate(int sampleRate, short [] audioData)
{
    int numSamples = audioData.length;
    int numCrossing = 0;
    for (int p = 0; p < numSamples-1; p++)
    {
        if ((audioData[p] > 0 && audioData[p + 1] <= 0) ||
                (audioData[p] < 0 && audioData[p + 1] >= 0))
        {
            numCrossing++;
        }
    }


    float numSecondsRecorded = (float)numSamples/(float)sampleRate;
    float numCycles = numCrossing/2;
    float frequency = numCycles/numSecondsRecorded;


    return frequency;
}

Answer 3

Your problem is right here:

Findfft();
for(int k=0;k<fftArray.length;k++) {
    freq = ((float)srate/(float) fftArray.length) *(float)k;
    runOnUiThread(new Runnable() {
        public void run() {
            disp.setText("The frequency is " + freq);
            if(freq>=15000) recording = false;
        }
    });
}

All this for loop does is go through your array of FFT values, convert the array index to a frequency in Hz, and print it.

If you want to output what frequency you're recording, you should at least look at the data in your array - the crudest method would be to calculate the square real magnitude and find the frequency bin with the biggest.

In addition to that, I don't think the FFT algorithm you're using does any precalculations - there are others that do, and seeing as you're developing for a mobile device, you might want to take CPU usage and power use into account.

JTransforms is one library that does use precalculation to lower CPU load, and its documentation is very complete.

You may also find useful information on how to interpret the data returned from the FFT at Wikipedia - no offense, but it looks like you're not quite sure what you're doing, so I'm giving pointers.

Lastly, if you're looking to use this app for musical notes, I seem to remember lots of people saying that an FFT isn't the best way to do that, but I can't remember what is. Maybe someone else can add that bit?