How can I detect and survive being “Slashdotted”?

Question

What's a good way to survive abnormally high traffic spikes?

My thought is that at some trigger, my website should temporarily switch into a "low bandwidth" mode: switch to basic HTML pages, minimal graphics, disable widgets that might put unnecessary load on the database, and so-on.

My thoughts are:

• Monitor CPU usage
• Monitor bandwidth
• Monitor requests / minute

Edit: I am familiar with options like caching, switching to static content or a content delivery network, and so on as a means to survive, so perhaps the question should focus more on how one detects when the website is about to become overloaded. (Although answers on other survival methods are of course still more than welcome.) Lets say that the website is running Apache on Linux and PHP. This is probably the most common configuration and should allow the maximum number of people to gain assistance from the answers. Lets also assume that expensive options like buying another server and load balancing are unavailable - for most of us at least, a mention on Slashdot is going to be a once-in-a-lifetime occurrence, and not something we can spend money preparing for.

Answer 1

It's worth mentioning that clever caching and low bandwidth modes will be useless if you simply don't have enough bandwidth on your connection, so make sure the connection to your server is fat enough. Don't host it on your home DSL connection, for example.

I speak from experience of being slashdotted. It's not fun when you can't access the Internet at all because thousands of people are simultaneously trying to download photos of a computer your housemate mounted inside a George Foreman grill. No amount of firewalling will save you.

Answer 2

You can also use Nagios to monitor the server health. Based on your requirements, at certain conditions, you can trigger an existing SQL file to switch modes for your website.

For example, add "UPDATE settings_table SET bandwidth = 'low';" into that SQL file and run it in mysql and do the opposite when the conditions get back to normal.

Answer 3

Use caching!

If you're using WordPress (for example), you can use something like WP-Super-Cache. If you're using regular PHP there are still a number of options you can use including memcache. Or you can just use regular squid proxy style caching.

Any caching you use will help bulletproof (or slashdot/digg-proof) your site :-)

Answer 4

There's simply no way to know whether or not your website will survive heavy loads unless you stress test it. Use something like siege and see where your performance problems lie. Does it grow in memory too quickly? Does it start slowing down with a bunch of concurrent connections? Does it start taking forever to access the database?

Once you know where the performance problems lie, then it becomes a matter of getting rid of them. Unfortunately, it's difficult to go into much more detail than that without knowing more about your particular situation, but keep in mind that you ARE talking about optimizations here. Thus, you should only act when you KNOW there are performance problems.

And I would argue that you're not necessarily just preparing for a once in a lifetime event. DOS attacks still happen, so it's good to have preparations in place even if your site doesn't get slashdotted.

The only thing that I can think of off the top of my head that will help you in almost all situations is if you gzip your content. That will save a lot of bandwidth and all modern browsers will support it without too much of a performance problem.

Answer 5

The real question is "What is the single most effective way to be Slashdotted"

If it's a real problem, redirect the traffic to my site.

Answer 6

About surviving you are right: switch or redirect the slashdotted link to a static html page without graphics. You might even want to put this page on an other webserver, so your original server will not take too much load.

I'd use a temporary redirection for this, and remove the redirection when the traffic wears off.

But how to detect this, this I'd like to know, too! Just counting the hits in the last few seconds might not be enough?