Code to Color Waveforms by Frequency [closed]

Question

I want to display audio waveforms color-coded in each part by the local frequency content. Basically exactly what Serato/Traktor or any other DJ software does where you can look at the sound and tell what frequencies are there. It looks like this:

So essentially, I will do an FFT to get frequencies at whatever bin-width I specify, but can anyone refer me to some code (preferably c) that would be useful in actually DRAWING it?

Answer 1

Let's try a real answer this time. :-)

The problem is too complex to give a complete solution with all the code in this space, but I will use pseudocode and assume you have some library that can window a block of samples and compute FFTs.

It's similar to building a waveform display. When you build your waveform display, you determine how many samples "fit" into a single horizontal pixel at the current zoom level, where they start given your X-scroll position, calculate the minimum and maximum sample value for that segment and gives you the min/max Y position for that waveform pixel. (This is actually a bit simplified, I've written waveform rendering code back in the day but this is a good approximation.)

To color your waves with frequency, you want to preproces the wave data using a short time FFT with smallish bins, and for each bin determine what the predominant frequency is and then map it to a color on the spectrum from red to violet.

Let's say your audio samples in an array called samples, here's the pseudo code.

// sample rate
float fS = 44100;

// size of frame for analysis, you may want to play with this 
float frameMsec = 10;

// samples in a frame
int frameSamples = (int)(fS / (frameMsec * 1000));

// how much overlap each frame, you may want to play with this one too
int overlapSamples = (frameSamples / 2); 

// number of samples in the sound file
int numSamples = ...; 

// input array of samples
float inSamples[] = ...;

// color to use for each frame
RGB outColors[] = new float[(numSamples / frameOverlap) + 1]; 

// scratch buffers
float tmpWindow[frameSamples];
float tmpFFT[frameSamples];

// helper function to apply a windowing function to a frame of samples
void calcWindow(float* dst, const float* src, int size);

// helper function to compute FFT
void fft(float* dst, const float* src, int size);

// find the index of array element with the highest absolute value
// probably want to take some kind of moving average of buf[i]^2
// and return the maximum found
int maxFreqIndex(const float* buf, int size);

// map a frequency to a color, red = lower freq -> violet = high freq
RGB freqToColor(int i);

for (int i = 0, outptr = 0; i < numSamples; i += frameOverlap, outptr++)
{
    // window another frame for FFT
    calcWindow(tmpWindow, &inSamples[i], frameSamples);

    // compute the FFT on the next frame
    fft(tmpFFT, tmpWindow, frameSamples);

    // which frequency is the highest?
    int freqIndex = maxFreqIndex(tmpFFT, frameSamples);

    // map to color
    outColor[outptr] = freqToColor(freqIndex);
}

This will give you an array of RGB which you can zoom in and out of as your zoom your waveform display in and out. As you zoom you probably want to average the RGB values of adjancent frames to give you an overall view.

I hope this helps.