Thrift large file in messages

Question

I am using thrift for binary protocol over tcp, for sending and reading files (maximum size of 64MB). Is thrift is capable for this?

I have thought using:

struct SomeMessage {
 1: byte data
}

How thrift is efficient about this? marshaling/unmarshaling/sending over the wire?

Answer 1

Thrift is capable, and the recommended data type is binary. Using byte would transfer only one signed 8-bit number, and list<byte> is way less efficient.

The question is, whether it makes sense to burden that load onto the Thrift service, or not. For example, with downloads it could make more sense to defer the transmission by passing a link or sth. similar to the client and have the (static) file being delivered by some other mechanism. That depends on your scenario, though, and it may not work when the client sends the data to the server.

Regarding overall efficiency, some good comparisons can be found on Google code. Again, if the performance you get suits your needs, depends on the problem and on the restrictions your are faced with. There is no finite answer on this.