C callbacks and non-Go threads

Question

1. How does one call Go code in C from threads that weren't created by Go?
2. What do I assign to a C function pointer such that threads not created by Go can call that pointer and enter into Go code?

Update0

• I don't want to use SWIG.
• The callbacks will be coming from threads Go hasn't seen before. Neither cgo/life nor anything in pkg/runtime demonstrates this behaviour AFAICT.

Answer 1

I'll assume you mean from C code compiled with gcc?

IIRC, this either can't be done or can't easily be done using 6g+cgo and friends. Go uses a different calling convention (as well as the segmented stacks and such).

However, you can write C code for [685]c (or even [685]a) and call into go easily using package·function() (you can even call methods IIRC). See the Source of the runtime package for examples.

Update:

Coming back to this question after the update, and giving it some more thought. This can't be done in a standard fashion using 6c or cgo. Especially because the threads are not started by the go runtime, the current implementation would fail. The scheduler would suddenly have a thread under its control that it does not know about; additionally, that thread would be missing some thread-local variables the go runtime uses for managing stacks and some other things. Also, if the go function returns a value (or several) the C code can't access it on the currently supported platforms, as go returns values on the stack (you could access them with assembly though). With these things in mind, I do believe you could still do this using channels. It would require your C code to be a little too intimate with the inner workings of the go runtime, but it would work for a given implementation. While using channels may not be the solution you're looking for, it could possibly fit more nicely with the concepts of Go than callbacks. If your C code reimplemented at least the sending methods in The channel implementation (that code is written for 6c, so it would have to be adapted for gcc most likely, and it calls the go runtime, which we've determined can't be done from a non-go thread), you should be able to lock the channel and push a value to it. The go scheduler can continue to manage it's own threads, but now it can receive data from other threads started in C.

Admittedly, it's a hack; I haven't looked close enough, but it would probably take a few other hacks to get it working (I believe the channels themselves maintain a list of the goroutines that are waiting on them [EDIT: confirmed: runtime·ready(gp);], so you'd need something in your go code to wake up the receiving channel or to warranty the go code won't receive on the channel until you've already pushed a value). However, I can't see any reason this can't work, whereas there are definite reasons that running code generated by 6g on a thread created in C can't.

My original answer still holds though: barring an addition to the language or runtime, this can't yet be done the way you'd like (I'd love to be proven wrong here).

Answer 2

You can do this, but the solution is relatively slow (about 22µs per call on my machine). The answer is for the C code to use C thread primitives to communicate with another goroutine that will actually run the callback.

I have created a Go package that provides this functionality: rog-go.googlecode.com/hg/exp/callback. There is an example package demonstrating its use here. The example demonstrates a call back to an arbitrary Go closure from a thread created outside of the Go runtime. Another example is here. This demonstrates a typical C callback interface and layers a Go callback on top of it.

To try out the first example:

goinstall rog-go.googlecode.com/hg/exp/example/looper
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/looper
gotest

To try out the second example:

goinstall rog-go.googlecode.com/hg/exp/example/event
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/event
gotest

Both examples assume that pthreads are available. Of course, this is just a stop-gap measure until cgo is fixed, but the technique for calling arbitrary Go closures in a C callback will be applicable even then.

Here is the documentation for the callback package:

PACKAGE

package callback
import "rog-go.googlecode.com/hg/exp/callback"

VARIABLES

var Func = callbackFunc

Func holds a pointer to the C callback function. When called, it calls the provided function f in a a Go context with the given argument.

It can be used by first converting it to a function pointer and then calling from C. Here is an example that sets up the callback function:

//static void (*callback)(void (*f)(void*), void *arg);
//void setCallback(void *c){
//  callback = c;
//}
import "C"
import "rog-go.googlecode.com/hg/exp/callback"

func init() {
    C.setCallback(callback.Func)
}

Answer 3

You can find a real-world application of rog's callback package in these bindings for the PortAudio audio I/O library: http://code.google.com/p/portaudio-go/. Might make it easier to understand..

(Thanks for implementing that, rog. It's just what I needed!)