Enabling compression on Heroku using python

Google now penalizes due to not being so mobile friendly. So in an effort to make things better, it recommends I compress a lot of my Javascript with Gzip or Deflate. I've seen some old recommendations on stack overflow, but there is nothing out of the box and I tried searching for add-ons, but as yet there doesn't appear to be anything that would do the trick. What is the least painful and robust of compressing or enabling gzip?

Here is what Google suggests I do:

Enable compression Compressing resources with gzip or deflate can reduce the number of bytes sent over the network. Enable compression for the following resources to reduce their transfer size by 420KiB (74% reduction).

I'm using Django if that makes it any easier.

Bottom Line Up Front - It's going to depend on the details of your app... Flask? Django? uWSGI? whitenoise and gunicorn seem to be the "go to" frameworks on Heroku, so that's what I used in the example below. It should translate to other frameworks.

Explication - The gist of the Google recommendation is about minimizing the number of bytes physically transferred from the server. There are several ways to do this, but among the highest impact, in no particular order -

Minify JavaScript and CSS
Merge those files together
Manipulate cache behavior
Compress the HTTP response body

The quoted recommendation deals with that last bit, and it's important to understand that compressing the response body is part of "content negotiation" in the HTTP specification - the browser doesn't just ask for a particular resource via the URL; it also provides hints about its preferred representation of that resource, e.g., what content type, how it's encoded, can it be sent in multiple "chunks", etc.

Thus, ideally, the layer of the application that handles HTTP should handle this particular task. In a typical application stack, that would mean a web server like Apache or nginx, in which the web server will proxy requests for specific, dynamic paths to your web framework, and handle the "static" content directly.

In Heroku, however, the HTTP layer is split between the platform itself and your application - the "routing mesh" acts as a reverse proxy, handling basic HTTP and HTTPS and enhancing the requests by injecting headers with proxy information, for example; everything else is up to your app. However, your "app" is fairly constrained, since you don't have free-reign to install nginx, etc.

Most web frameworks (Django, Flask, Rails, Play!, etc. etc.) are highly generalized, and can work in conjunction with an external web server (recommended for production) or can work independently, providing their own, usually lightweight web servers (recommended for development). The frameworks also pair well with "containers" that provide both the run-time environment for the application and the heavy lifting at the HTTP layer (uWSGI, Gunicorn, Rack, etc.)

This is the option to go for with Heroku. Although I have the most experience with uWSGI, the example below is for Flask + Gunicorn + WhiteNoise (the preferred library for serving static files on Heroku in Python). Note that WhiteNoise also works with Django, so adapting this should be trivial, should Django be your framework of choice. So, all that exposition results in two pretty simple steps to get started:

Add whitenoise to your requirements.txt
Modify the WSGI application to have WhiteNoise "wrap" your application.

For example:

from flask import Flask
from whitenoise import WhiteNoise

flapp = Flask(__name__)
#use a subdirectory for root, otherwise, the actual .py files can be served...
app = WhiteNoise(flap, root='./static/')

#define your routes:
@flapp.route('/')
def home_page():
    #etc. etc.

This will get you gzip'd content if the client sends an "Accept-Encoding: gzip" header. There are many, many other levers and knobs to pull and tweak, but this is a starting point. Eventually, you'll worry about CPU overhead and want to pre-compress the files; or you may decide that off-loading static files is the way to go.

To verify, use a tool like cURL to grab a static file:

curl -i -H "Accept-Encoding: gzip" http://yourapp.herokuapp.com/path/to/static

The -i flag should print out headers, which will show you the details of how the request was served. Note the `Content-Encoding

HTTP/1.1 200 OK
Connection: keep-alive
Server: gunicorn/19.3.0
Date: Wed, 20 May 2015 15:33:35 GMT
Last-Modified: Wed, 20 May 2015 15:26:06 GMT
Content-Type: text/html; charset="utf-8"
Cache-Control: public, max-age=60
Access-Control-Allow-Origin: *
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 662
Via: 1.1 vegur

Hope this helps...