C. Loop compression + send (gzip) ZLIB

Question

I'm currently building an HTTP server in C.

Please mind this piece of code :

  #define CHUNK 0x4000
  z_stream strm;
  unsigned char out[CHUNK];
  int ret;
  char buff[200];

  strm.zalloc = Z_NULL;
  strm.zfree = Z_NULL;
  strm.opaque = Z_NULL;

  int windowsBits = 15;
  int GZIP_ENCODING = 16;

  ret = deflateInit2(&strm, Z_BEST_SPEED, Z_DEFLATED, windowsBits | GZIP_ENCODING, 1, 
                     Z_DEFAULT_STRATEGY);
  fill(buff); //fill buff with infos

  do {
  strm.next_in = (z_const unsigned char *)buff;
  strm.avail_in = strlen(buff);
      do {
        strm.avail_out = CHUNK;
        strm.next_out = out;
        ret = deflate(&strm, Z_FINISH);
       } while (strm.avail_out == 0);

  send_to_client(out); //sending a part of the gzip encoded string
  fill(buff);
  }while(strlen(buff)!=0);

The idea is : I'm trying to send gzip'ed buffers, one by one, that (when they're concatened) is a whole body request.

BUT : for now, my client (a browser) only get the infos of the first buffer. No errors at all though.

How do I achieve this job, how to gzip some buffers inside a loop so I can send them everytime (in the loop) ?

Answer 1

First off, you need to do something with the generated deflate data after each deflate() call. Your code discards the compressed data generated in the inner loop. From this example, after the deflate() you would need something like:

        have = CHUNK - strm.avail_out;
        if (fwrite(out, 1, have, dest) != have || ferror(dest)) {
            (void)deflateEnd(&strm);
            return Z_ERRNO;
        }

That's where your send_to_client needs to be, sending have bytes.

In your case, your CHUNK is so much larger than your buff, that loop is always executing only once, so you are not discarding any data. However that is only happening because of the Z_FINISH, so when you make the next fix, you will be discarding data with your current code.

Second, you are finishing the deflate() stream each time after no more than 199 bytes of input. This will greatly limit how much compression you can get. Furthermore, you are sending individual gzip streams, for which most browsers will only interpret the first one. (This is actually a bug in those browsers, but I don't imagine they will be fixed.)

You need to give the compressor at least 10's to 100's of Kbytes to work with in order get decent compression. You need to use Z_NO_FLUSH instead of Z_FINISH until you get to your last buff you want to send. Then you use Z_FINISH. Again, take a look at the example and read the comments.