I have a large file of size 500 mb to compress in a minute with the best possible compression ratio. I have found out these algorithms to be suitable for my use.
- lz4
- lz4_hc
- snappy
- quicklz
- blosc
Can someone give a comparison of speed and compression ratios between these algorithms?
Yann Collet's lz4, hands down.
This migth help you:
(lz4 vs snappy) http://java-performance.info/performance-general-compression/
(benchmarks for lz4, snappy, lz4hc, blosc) https://web.archive.org/web/20170706065303/http://blosc.org:80/synthetic-benchmarks.html
(now not available on http://www.blosc.org/synthetic-benchmarks.html)
If you are only aiming for high compression density, you want to look at LZMA and large-window Brotli. These two algorithms give the best compression density from the widely available open-sourced algorithms. Brotli is slower at compression, but ~5x faster at decompression.