With Heroku, how does one AUTO scale up in terms of web dynos when it is needed? Say we get a surge of 100 concurrent users every 2-3 minutes. If our app is stuck on 5-6 web dynos. We are screwed.
Second, I wouldn't be able to monitor traffic 24 hours to determine whether a scale up or down is required.
So far, I've seen http://hirefireapp.com/ and http://www.heroscale.com/ Any suggestions about these two?
The reason heroku don't do this natively is that it's an incredibly complex problem to solve.
For instance, imagine your scenario above, you suddenly start seeing a queue forming and want to ramp the dynos. You crank on ten more. However, it's not a dyno problem, your database is running slow, so now you've got more dynos all sat waiting for the database which now has even more demand placed on it.
Whilst there are auto-scaling products out there, I've not tried any of them, and fully believe that at the moment only a human can make the correct call on scaling. Your mileage may vary.
I have found in the past that setting the resources to an expected usage level (which may be above the current usage) tends to work best, excluding massive traffic influx (such as being on Hacker News etc)
I built HireFire and would like to share some up-to-date information:
HireFire autoscales both your web- and worker dynos using our dyno managers. We currently support the following metric sources:
HireFire (Job Queue)
Autoscales your worker-based dyno based on the queue size of your jobs. Integration for Ruby and Python applications can be done easily using a 1st or 3rd party library. Any other language and/or framework can be integrated easily without a library as well.
You're able to configure any number of dyno managers for a given application at no extra cost, meaning that you're not limited to a single "worker" entry in your Procfile. This (optionally) allows you to schedule work more efficiently by having for example one Procfile entry per queue and having HireFire scale each individual queue independently.
Heroku Logplex
The Logplex (Logdrain) strategy allows HireFire to consume your logs in order to parse Heroku-emitted metric data which we then use to autoscale. Metrics include Response Time, Connect Time and Load. In addition to that we support Queue Time which can easily be added by installing our library. Or, you can write the minimal amount of code yourself to push the necessary data to the logdrain.
This abstract approach (excluding Queue Time) requires no code changes and works with any language/framework. Just setup a Logdrain via de Heroku CLI and you're set.
For metric aggregation, you can choose between average and (any) percentile.
New Relic
We integrate with New Relic. If you're already using it you can hook it up to HireFire and use their metrics (apdex, response time and rpm) to autoscale your web dynos.
If you have any questions, don't hesitate to get in touch!
A lot of my friends in the Rails community use Rails Autoscale for autoscaling on Heroku. Here's how it works:
Rails Autoscale provides a tiny Rack middleware that captures this timing and periodically reports it back to the Rails Autoscale service. This is similar to how New Relic works at a fraction of the size.
The autoscaling that Heroku provides natively is only available on their Performance tier (which starts at $250 / month per dyno). If you're using the Hobby or Standard plans, you'll need to find a 3rd party solution.
One thing I like about Rails Autoscale is it scales your app up and down automatically, based on request queueing.
It's a nice feature, especially for peace of mind. If you're sleeping, and get a traffic spike, you can't manually adjust the number of dynos. Having a tool that can scale up automatically is nice insurance.
Heroku just launched a new addon that does auto scaling. Web dynos only right now though.
Check out this thread https://stackoverflow.com/a/14075781/484689
I wrote a Heroku auto scaling engine called Heroku Vector. It allows you to scale web and sidekiq dynos based on the volume of traffic you receive (instead of waiting for latencies in response time):
https://github.com/wpeterson/heroku-vector
You can run it as a stand-alone dyno process.
As of January 2017, Heroku formally supports autoscaling.
Autoscaling is easy to set up and use, and it recommends a p95 threshold based on your app’s past 24 hours of response times. Response-based autoscaling ensures that your web dyno formation is always sized for optimal efficiency, while capping your costs based on limits you set. Autoscaling is currently included at no additional cost for apps using Performance and Private web dynos.
Here are the docs: https://devcenter.heroku.com/articles/scaling#autoscaling
Here is the announcement: https://blog.heroku.com/heroku-autoscaling
