Best approach for (cross-platform) real-time data

Question

I've been wondering how to do "true" (semi) real-time data streaming with PHP.

Possible applications: chat rooms, auctions, games, etc.

By "true", I mean the data is not just written somewhere and polled continuously, but actually streaming to the client somehow.

By "semi", I mean it's okay if only the stream from the server to the client is real-time, and messages from the client to the server are not.

For the communication between the client and server, I'd like to stick with plain HTTP (AJAX) rather than some other protocol.

Streaming to the client with HTTP is possible by manually flushing the output buffer.

The question is what to connect that script to on the server-side?

And once it's connected, to do a blocking read, rather than polling for changes.

The shared memory (shmop) extension would work, but it's not cross-platform.

Perhaps memcached would work? But I'm not sure if there's a way to do a blocking read, so it comes down to polling again - although I'm sure memcached is pretty fast, I just don't like the idea of continuous polling.

Any ideas?

Answer 1

PHP is not well suited for implementing realtime data streaming. PHP is very slow, and is not designed to build multi-threaded applications. You'd be better off implementing a full blown socket server in a language like Python or Java.

That said, I would highly recommend checking out NodeJS: http://nodejs.org/

It uses an asynchronous event based model for I/O, instead of having threads block in an event loop. NodeJS servers are written in Javascript. NodeJS is fast, scales, and has a low learning curve.

Clients would connect to a NodeJS HTTP server using long polling Ajax requests. PHP could connect to NodeJS directly and push notifications. Or PHP could write to a message queue, or database, memcache etc, and NodeJS would poll those data stores for updates, and send new messages to clients.

You would possibly need to write your own daemon to serve as a go between from NodeJS to MySQL, memcached, etc. when polling for updates. NodeJS would keep a socket open with a daemon process. The daemon process would poll data stores for updates, and send the updates to NodeJS. A NodeJS HTTP server would then send those updates to clients.

See this tutorial for implementing a realtime Twitter stream: http://net.tutsplus.com/tutorials/javascript-ajax/learning-serverside-javascript-with-node-js/

Answer 2

If you want to communicate between PHP and another language (for instance a C++ application) you might want to look into Apache Thrift (http://thrift.apache.org/). Apache Thrift is widely used at Facebook for "scalable cross-language services development".

Edit: I would probably use Apache Thrift to communicate with a Twisted application listening on port 80 and have the browsers connect to the Twisted application using either long polling or a websocket. You might want to also look into Socket.IO, it's a cross-browser web socket implementation which is made for realtime applications.

Basically you would have your application push to your Twisted web server using Thrift and you would then pass the message to any open connections.

• Christian

Answer 3

If you're using HTML and Javascript, then you want WebSockets. If it's Flash or anything else, then normal TCP sockets.

The idea for either is that you run a server file (written in PHP), that waits for connections. Once it's connected to one or more clients, data can be pushed both ways. There are a few PHP WebSocket projects out there. Check out this:

http://code.google.com/p/phpwebsocket

There's also a framework called Skeleton that I've been contributing to that has a WebSocket server library built in. Still in the unstable stages, though.

http://code.google.com/p/skeleton

Unfortunately, WebSockets are still a new technology, so they're not supported universally. As @Christian mentioned, you may want to use the Socket.IO library.