MIDI over TCP/IP/UDP to be received inUnity3D with

Question

I am working in Unity3D Standard Edition. For those that don't know, it is a 3D games engine which supports C#/.NET scripting (so it has access to the .NET 2.0 API).

I have created my own synthesiser. (Currently it is responding to note-ON events which are generated by a MIDI sequencer munching through a MIDI file.)

Now I wish to catch note-ON events from a MIDI keyboard.

I'm intending to deploy my app to iOS at the very least, hopefully other operating systems to follow. So target scenarios are:

• someone running my app on an iPhone, they have a MIDI keyboard connecting into their Windows machine
• someone running my app on an android tablet, they have a MIDI keyboard connecting into their Linux machine
• someone running my app on OS X, they have a MIDI keyboard connecting into OS X

The last one is the situation I'm in, so if anyone has a solution for that one that doesn't extend, I am still very interested in it -- that would at least allow me to get a functional prototype together.

There exists one asset, MIDI Unified, which will connect with a MIDI device. However, it requires the PRO version. This is because PRO version allows native plug-ins.

I believe it should be possible to get MIDI into Unity without using native plug-ins, but instead over the network connection.

(Note that I'm not saying Internet connection, Ethernet connection, TCP/IP, UDP, HTTP, as I'm not completely sure what the right term to use would be.)

It appears there are utilities for sending MIDI signals over networks: ipMIDI will do this for Windows and OS X. There is probably something for doing it in Linux.

OSX has an "Audio MIDI setup" utility which I have been told allow MIDI signals to be channelled from the device to localhost. (maybe this is why ipMIDI for OS X is free?)

EDIT: I've just discovered that Audio MIDI setup implements RTP MIDI, which appears to be the best standard for beaming MIDI over a network (it copes with lossy networks).

So I'm pretty sure the task becomes: how to implement RTP MIDI in C#/.NET?

But maybe there is some easier (but less powerful) solution, for example, some solution that only works for and receiving Note-ON MIDI messages from localhost on OSX.

I would be very happy if I could get a basic solution for of the first generation of my app, and then subsequently replaced this with a robust component when I have the means.

So that is as far as I have got, can anyone tidy this up all push it forwards?

π

EDIT: http://u3d.as/content/sta-blockhead/websocket-sharp-for-unity/4X4 Could this help?

Answer 1

I've done something similar in the past. What you could do is:

Install a Midi loopback device (driver) on your computer. There are many of those, for example: http://www.tobias-erichsen.de/software/loopmidi.html

Then design a simple Application that listens on the Virtual midi port from Loopback device and converts anything received from Virtual MIDI port to data packet that are sent over TCP/IP connection. TCP/IP connection will be terminating on your Unity Script or something else that supports TCP/IP and able to play MIDI. Protocol is quite simple 3 bytes - Note On, Note Number, Volume. (usually Volume 0 means note off). That's about how you could implement this.

Hope this helps...

Answer 2

Python is truly wonderful!

This script sends a UDP packet whenever a MIDI note on/off occurs. I can catch the UDP packets in Unity3D.

#!/usr/bin/python

# https://github.com/superquadratic/rtmidi-python/
import rtmidi_python as rtmidi

import time

import socket

def callback( data, time_stamp ):
    event, note, vel = data

    if event == 144: # note on/off
        endpoint = ( "127.0.0.1", 6500 )

        MESSAGE = "%d, %f" % ( note, float(vel) / 127.0 )

        print MESSAGE

        udp_socket = socket.socket( socket.AF_INET, socket.SOCK_DGRAM )

        udp_socket.sendto( 
            MESSAGE, 
            endpoint
            )

def main( ):
    midi_in = rtmidi.MidiIn( )

    midi_in.callback = callback

    midi_in.open_port( 0 )

    # do something else here (but don't quit)
    while True: 
        time.sleep( 0.001 )

if __name__ == '__main__': 
    main()