Serving a downloadable file (huge) from re

Question

I am trying to have my server serving a huge file from another server. However, to protect my credential against that remote server, I can't simply redirect the requester to the file url; on the other hand, although using StreamingHttpResponse doesn't add content to memory or hardrive, it does takes more time - the server still need to download the file from the remote then serve it, which doubles up the response time. Is there any way I can "redirect" content of the remote file, without actually downloading it? Thanks.

Answer 1

you can read the file and stream it at the same time.

look at the code in Django 1.5 - using the new StreamingHttpResponse - all you need to do is put the code that reads from the other server into the generator. each time you read some data from the socket, write it to the streaming output.

Answer 2

If you're using Amazon CloudFront, they solve this issue by a concept called "Signed URLs". Basically, you use use public key cryptography to generate a one-time-use URL; you can specify a time range that the URL is only usable for and that only a certain IP Address can download the content. Other cloud providers may also have similar features.

If you're not using CloudFront but you have control over the download server, you can build similar infrastructure. Basically, you sign, using your private key, a message containing a timestamp, the name of the file to download, and the download policy (e.g. expiry date and IP range), and the remote server uses your public key to validate that the message actually comes from your server and then enforces the policy.

This is the most efficient way to serve large files from remote server securely as your server wouldn't need to download the file first to your server.