I am trying my hands at Audio Processing in python with this Beat Detection algorithm. I have implemented the first (non-optimized version) from the aforementioned article. While it prints some results, I have no way to detect whether it works with some accuracy or not as I do not know how to play sound with it.
Currently, I am using Popen
to asynchronously start my media player with the song before going into the computation loop, but I am not sure if this strategy works and is giving synchronous results.
#!/usr/bin/python
import scipy.io.wavfile, numpy, sys, subprocess
# Some abstractions for computation
def sumsquared(arr):
sum = 0
for i in arr:
sum = sum + (i[0] * i[0]) + (i[1] * i[1])
return sum
if sys.argv.__len__() < 2:
print 'USAGE: wavdsp <wavfile>'
sys.exit(1)
numpy.set_printoptions(threshold='nan')
rate, data = scipy.io.wavfile.read(sys.argv[1])
# Beat detection algorithm begin
# the algorithm has been implemented as per GameDev Article
# Initialisation
data_len = data.__len__()
idx = 0
hist_last = 44032
instant_energy = 0
local_energy = 0
le_multi = 0.023219955 # Local energy multiplier ~ 1024/44100
# Play the song
p = subprocess.Popen(['audacious', sys.argv[1]])
while idx < data_len - 48000:
dat = data[idx:idx+1024]
history = data[idx:hist_last]
instant_energy = sumsquared(dat)
local_energy = le_multi * sumsquared(history)
print instant_energy, local_energy
if instant_energy > (local_energy * 1.3):
print 'Beat'
idx = idx + 1024
hist_last = hist_last + 1024 # Right shift history buffer
p.terminate()
What modification/additions can I make to the script in order to get audio output and the algorithm (console) output in a time synchronised manner? i.e When console outputs result for a particular frame, that frame must be playing on the speakers.
Working beat detection code (NumPy / PyAudio)
If you are using NumPy this code might help. It assumes the signal (read with PyAudio) is 16-bit wide Int. If that is not the case change or remove the signal.astype() and adjust the normalization-divider (max int16 here).
The PyAudio examples for wav read or mic record will give you the needed signal data. Create a NumPy array efficiently with
frombuffer()
A Simpler, Non-Realtime Approach
I'm not optimistic about synchronizing console output with realtime audio. My approach would be a bit simpler. As you read through the file and process it, write the samples out to a new audio file. Whenever a beat is detected, add some hard-to-miss sound, like a loud, short sine tone to the audio you're writing. That way, you can aurally evaluate the quality of the results.
Synthesize your beat indicator sound:
In your
while
loop, add the testsignal to the input frame if a beat is detected, and leave the frame unaltered if no beat is detected. Write those frames out to a file and listen to it to evaluate the quality of the beat detection.This is the approach used by the aubio library to evaluate beat detection results. See the documentation here. Of particular interest is the documentation for the
--output
command line option:Optimization
Since numpy is already a dependency, use its capabilities to speed up your algorithm. You can rewrite your
sumsquared
function as:Getting rid of the Python for-loop and pushing those calculations down into C code should give you a speed improvement.
Also, take a look at this question or this question to get an idea of how you might vectorize the local to instantaneous energy comparisons in the
while
loop, using thenumpy.lib.stride_tricks
method.A good bet would be to try portaudio (pyaudio) to get the data live, then you should be able to see if it matches.
Here's a nice example using fft from the mic with pyaudio:
http://www.swharden.com/blog/2010-03-05-realtime-fft-graph-of-audio-wav-file-or-microphone-input-with-python-scipy-and-wckgraph/