Very large uploads with PHP

Question

I want to allow uploads of very large files into our PHP application (hundred of megs - 8 gigs). There are a couple of problems with this however.

Browser:

• HTML uploads have crappy feedback, we need to either poll for progress (which is a bit silly) or show no feedback at all
• Flash uploader puts entire file into memory before starting the upload

Server:

• PHP forces us to set post_max_size, which could result in an easily exploitable DOS attack. I'd like to not set this setting globally.
• The server also requires some other variables to be there in the POST vars, such as an secret key. We'd like to be able to refuse the request right away, instead of after the entire file is uploaded.

Requirements:

• HTTP is a must.
• I'm flexible with client-side technology, as long as it works in a browser.
• PHP is not a requirement, if there's some other technology that will work well on a linux environment, that's perfectly cool.

Answer 1

You can set the post_max_size for just scripts in 1 directory. Place your upload script there, and allow only that script to handle large sizes. It's still possible for that script to be attacked with large/useless files, but it avoids setting it globally.

Use that with APC and you might be able to work out something good: IBM Developer works article on APC

Answer 2

I would look into FTP, SSH or SCP this allows you to upload a large file and still have access control over the file as well. This might take a little longer to implement but its probably the most secure way I could think of.

Answer 3

I've had success with uploadify, and I would recommend it. It's a jQuery/Flash script that handles large uploads, and you can pass extra parameters to it (like the secret key). To solve the server-side issues, simply use the following code. The changes take affect just for the script they're called in:

//Check to see if the key is there
if(!isset($_POST['secret_key']) || !isValid($_POST['secret_key']))
{
    exit("Invalid request");
}
function isValid($key)
{
    //Put your validation code here.
}

//This line changes the timeout.
//Give it a value in seconds (3600 = 1 hour)
set_time_limit(3600);

//Set these amounts to whatever you need.
ini_set("post_max_size","8192M");
ini_set("upload_max_filesize","8192M");

//Generally speaking, the memory_limit should be higher
//than your post size.  So make sure that's right too.
ini_set("memory_limit","8200M");

EDIT In response to your comment:

Given what you've said, I'm afraid you may not be able to meet your requirements over http. All of the solutions out there are code that add features to http that it was never designed for.

Like you said yourself, it's a simple protocol. Apart from writing your own client software that runs outside of the browser, a java applet, or using a different protocol (like FTP, which was designed for this), you might not get what you want.

I've done the best I could within the given constraints. Sorry I couldn't do better.

Answer 4

Take a look at jumploader.com

A good java-applet for uploading.

I've used it for uploading images and it works fine. Haven't tried with bigger files than 10MB, but i should work for really big files too.

Answer 5

Try this: http://www.simple2ftp.com uses a Java based FTP applet from within a clever PHP application wrapper.

Answer 6

How about a Java applet? That's how we had to do it at a company I previously worked for. I know applets suck, especially in this day and age with all our options available, but they really are the most versatile solution to desktop-like problems encountered in web development. Just something to consider.