Getting Multiple Audio Inputs in Processing

Question

I'm currently writing a Processing sketch that needs to access multiple audio inputs, but Processing only allows access to the default line in. I have tried getting Lines straight from the Java Mixer (accessed within Processing), but I still only get the signal from whichever line is currently set to default on my machine.

I've started looking at sending the sound via OSC from SuperCollider, as recommended here. However, since I'm very new to SuperCollider and their documentation and support is more focused on generating sound than on accessing inputs, my next step will probably be to play around with Beads and Jack, as suggested here.

Does anyone have (1) other suggestions, or (2) concrete examples of getting multiple inputs from either SuperCollider or Beads/Jack to Processing?

Thank you in advance!

Edit: The sound will be used to power custom music visualizations (think the iTunes visualizer, but much more song specific). We have this working with multiple mp3s; now what I need is to able to get a float[] buffer from each mic. Hoping to have 9 different mics, though we'll settle for 4 if that is more doable.

For hardware, at this point, we are just using mics and XLR to USB cables. (Have considered a pre-amp, but so far this has been sufficient.) I am currently on Windows, but I think that we will ultimately switch to a Mac.

Here was my attempt with just Beads (it works fine for the laptop, since I do that one first, but the headset buffer has all 0's; if I switch them and put the headset first, the headset buffer will be correct, but the laptop will contain all 0's):

void setup() {
    size(512, 400);

    JavaSoundAudioIO  headsetAudioIO = new JavaSoundAudioIO();
    JavaSoundAudioIO  laptopAudioIO = new JavaSoundAudioIO();

    headsetAudioIO.selectMixer(5);
    headsetAudioCon  = new AudioContext(headsetAudioIO);

    laptopAudioIO.selectMixer(4);
    laptopAudioCon  = new AudioContext(laptopAudioIO);

    headsetMic  = headsetAudioCon.getAudioInput();
    laptopMic  = headsetAudioCon.getAudioInput();
} // setup()

void draw() {
    background(100,0, 75);

    laptopMic.start();
    laptopMic.calculateBuffer();
    laptopBuffer   = laptopMic.getOutBuffer(0);

    for (int j = 0; j < laptopBuffer.length - 1; j++)
    {
        println("laptop; " + j + ": " + laptopBuffer[j]);
        line(j, 200+laptopBuffer[j]*50, j+1, 200+laptopBuffer[j+1]*50);
    }
    laptopMic.kill();

    headsetMic.start();
    headsetMic.calculateBuffer();

    headsetBuffer  = headsetMic.getOutBuffer(0);


    for (int j = 0; j < headsetBuffer.length - 1; j++)
    {
        println("headset; " + j + ": " + headsetBuffer[j]);
        line(j, 50+headsetBuffer[j]*50, j+1, 50+headsetBuffer[j+1]*50);
    }

    headsetMic.kill();
} // draw()

My attempt at adding Jack contains this line:

ac = new AudioContext(new AudioServerIO.Jack(), 44100, new IOAudioFormat(44100, 16, 4, 4));

but I get the error:

Jun 22, 2016 9:17:24 PM org.jaudiolibs.beads.AudioServerIO$1 run
SEVERE: null
org.jaudiolibs.jnajack.JackException: Can't find native library
    at org.jaudiolibs.jnajack.Jack.getInstance(Jack.java:428)
    at org.jaudiolibs.audioservers.jack.JackAudioServer.initialise(JackAudioServer.java:102)
    at org.jaudiolibs.audioservers.jack.JackAudioServer.run(JackAudioServer.java:86)
    at org.jaudiolibs.beads.AudioServerIO$1.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: Unable to load library 'jack': Native library (win32-x86-64/jack.dll) not found in resource path ([file:/C:/Users/...etc...)

And when I'm in Jack, I don't see my mic (which seems like a huge red flag to me, though I am completely new to Jack). Should this AudioContext show up as an Input in Jack? Or vice versa -- find my mic there first and then get it from Jack to Processing?

(Forgive my inexperience, and thank you again! My lack of knowledge in Jack makes me wonder if I should revisit SuperCollider instead...)

Answer 1

I had the same issue a few years ago and I used a combination of JACK, JNAJack and Beads. You can follow this Beads Google Group thread for more details.

At the that time I had to use this version of Beads (2012-04-23), but I hope those changes probably made it into the main project by now.

For reference, here is the basic class I used:

import java.util.Arrays;

import org.jaudiolibs.beads.AudioServerIO;

import net.beadsproject.beads.analysis.featureextractors.FFT;
import net.beadsproject.beads.analysis.featureextractors.PowerSpectrum;
import net.beadsproject.beads.analysis.segmenters.ShortFrameSegmenter;
import net.beadsproject.beads.core.AudioContext;
import net.beadsproject.beads.core.AudioIO;
import net.beadsproject.beads.core.UGen;
import net.beadsproject.beads.ugens.Gain;
import processing.core.PApplet;


public class BeadsJNA extends PApplet {

    AudioContext ac;
    ShortFrameSegmenter sfs;
    PowerSpectrum ps;

    public void setup(){
        //defining audio context with 6 inputs and 6 outputs - adjust this based on your sound card / JACK setup
        ac = new AudioContext(new AudioServerIO.Jack(),512,AudioContext.defaultAudioFormat(6,6));

        //getting 4 audio inputs (channels 1,2,3,4)
        UGen microphoneIn = ac.getAudioInput(new int[]{1,2,3,4});
        Gain g = new Gain(ac, 1, 0.5f);
        g.addInput(microphoneIn);
        ac.out.addInput(g);

        println("no. of inputs:  " + ac.getAudioInput().getOuts()); 

        //test get some FFT power spectrum data form the 
        sfs = new ShortFrameSegmenter(ac);
        sfs.addInput(ac.out);
        FFT fft = new FFT();
        sfs.addListener(fft);
        ps = new PowerSpectrum();
        fft.addListener(ps);
        ac.out.addDependent(sfs);

        ac.start();
    }
    public void draw(){
        background(255);
        float[] features = ps.getFeatures();
        if(features != null){
            for(int x = 0; x < width; x++){
              int featureIndex = (x * features.length) / width;
              int barHeight = Math.min((int)(features[featureIndex] *
                                                height), height - 1);
              line(x, height, x, height - barHeight);
            } 
        }
    }

    public static void main(String[] args) {
        PApplet.main(BeadsJNA.class.getSimpleName());
    }

}