Spent yesterday evening reading about and implementing Snappy Compression Algorithm and it is lit ⚡ and here’s how it works …
Phase 1: Dictionary Creation
Before compressing the data, Snappy creates a dictionary based on a sample of the input data. This dictionary consists of sequences of bytes that occur frequently in the input data. The dictionary is used to replace these sequences with shorter codes during compression, which improves the compression ratio.
Phase 2: Blocks
Snappy divides the input data into blocks of up to 64KB in size. Each block is compressed separately, which makes it possible to decompress individual parts of the data without having to decompress the entire file.
Phase 3: Huffman coding
Snappy uses Huffman coding to compress the data within each block. Huffman coding assigns shorter codes to more frequently occurring symbols. Snappy uses a pre-defined set of Huffman codes that is optimized for the type of data being compressed.
Phase 4: Compressed output
The compressed data is output in a block-based format that includes information about the size of each block, the length of the uncompressed data, and the contents of the dictionary.
Phase 5: Decompression
Snappy reverses the compression process. It reads the block headers, decompresses each block using the Huffman codes and dictionary, and concatenates the decompressed blocks to produce the original data.
Snappy is designed to be very fast at compressing and decompressing data, making it an excellent choice for real-time applications where performance is critical.
✨ The best way to understand Snappy is by re-writing it in your favorite programming language. I used Golang.
Highly recommended.