How to send binary file over Web Sockets with Rail

Question

I have a Rails application where users upload Audio files. I want to send them to a third party server, and I need to connect to the external server using Web sockets, so, I need my Rails application to be a websocket client.

I'm trying to figure out how to properly set that up. I'm not committed to any gem just yet, but the 'faye-websocket' gem looks promising. I even found a similar answer in "Sending large file in websocket before timeout", however, using that code doesn't work for me.

Here is an example of my code:

@message = Array.new
EM.run {
    ws = Faye::WebSocket::Client.new("wss://example_url.com")

    ws.on :open do |event|
      File.open('path/to/audio_file.wav','rb') do |f|
        ws.send(f.gets)
      end
    end

    ws.on :message do |event|
      @message << [event.data]
    end

    ws.on :close do |event|
      ws = nil
      EM.stop
    end
}

When I use that, I get an error from the recipient server:

No JSON object could be decoded

This makes sense, because the I don't believe it's properly formatted for faye-websocket. Their documentation says:

send(message) accepts either a String or an Array of byte-sized integers and sends a text or binary message over the connection to the other peer; binary data must be encoded as an Array.

I'm not sure how to accomplish that. How do I load binary into an array of integers with Ruby?

I tried modifying the send command to use the bytes method:

File.open('path/to/audio_file.wav','rb') do |f|
    ws.send(f.gets.bytes)
end

But now I receive this error:

Stream was 19 bytes but needs to be at least 100 bytes

I know my file is 286KB, so something is wrong here. I get confused as to when to use File.read vs File.open vs. File.new.

Also, maybe this gem isn't the best for sending binary data. Does anyone have success sending binary files in Rails with websockets?

Update: I did find a way to get this working, but it is terrible for memory. For other people that want to load small files, you can simply File.binread and the unpack method:

ws.on :open do |event|
  f = File.binread 'path/to/audio_file.wav'
  ws.send(f.unpack('C*'))
end

However, if I use that same code on a mere 100MB file, the server runs out of memory. It depletes the entire available 1.5GB on my test server! Does anyone know how to do this is a memory safe manner?

Answer 1

Here's my take on it:

# do only once when initializing Rails:
require 'iodine/client'
Iodine.force_start!

# this sets the callbacks.
# on_message is always required by Iodine.
options = {}
options[:on_message] = Proc.new do |data|
   # this will never get called
   puts "incoming data ignored? for:\n#{data}"
end
options[:on_open] = Proc.new do
   # believe it or not - this variable belongs to the websocket connection.
   @started_upload = true
   # set a task to send the file,
   # so the on_open initialization doesn't block incoming messages.
   Iodine.run do
      # read the file and write to the websocket.
      File.open('filename','r') do |f|
         buffer = String.new # recycle the String's allocated memory
         write f.read(65_536, buffer) until f.eof?
         @started_upload = :done
      end
      # close the connection
      close
   end
end
options[:on_close] = Proc.new do |data|
   # can we notify the user that the file was uploaded?
   if @started_upload == :done
        # we did it :-)
   else
        # what happened?
   end
end

# will not wait for a connection:
Iodine::Http.ws_connect "wss://example_url.com", options
# OR
# will wait for a connection, raising errors if failed.
Iodine::Http::WebsocketClient.connect "wss://example_url.com", options

It's only fair to mention that I'm Iodine's author, which I wrote for use in Plezi (a RESTful Websocket real time application framework you can use stand alone or within Rails)... I'm super biased ;-)

I would avoid the gets because it's size could include the whole file or a single byte, depending on the location of the next End Of Line (EOL) marker... read gives you better control over each chunk's size.