huffman-coding | Huffman coding and Adaptive Huffman coding algorithms | Cryptography library

The zlib library supports dictionaries with the zlib (not gzip) format. See deflateSetDictionary() and inflateSetDictionary().

There is nothing special about the construction of a dictionary. All it is is 32K bytes of strings that you believe will occur often in the data you are compressing. You should put the most common strings at the end of the 32K.

It is generating the codes correctly, but then printing them incorrectly. It is leaving off the leading zero bits of the codes that have them. They should have prepended the necessary zero bits after converting the number to a string of digits.

If you replace the line that prints the code with this:

Yes, there are several ways to generate prefix-free codes dynamically.

As you suggested, it would be conceptually simple to start with some default frequency, track the frequencies of the letters used so far, and for every letter decoded, increment that letter's count and then re-build a Huffman tree from all the counts. (potentially completely changing the tree after each letter). That would require a lot of work for each letter and be very slow -- and yet there are a couple of adaptive Huffman coding algorithms that effectively do the same thing -- using clever algorithms that do much less work, and so are faster.

Many other data compression algorithms also generate prefix-free codes dynamically much faster than any adaptive Huffman algorithm, at a small sacrifice of compression -- such as Polar codes or Engel coding or universal codes such as Elias delta coding.

The arithmetic coding data compression algorithm is technically not a prefix-free code, but typically gives slightly better compression (but runs slower) than either static Huffman coding or adaptive Huffman coding. Arithmetic coding is generally implemented adaptively, tracking the frequencies of all the letters used so far. (Many arithmetic coding implementations track even more context -- if the previous letter was a "t", it remembers that the most-frequent letter in this context is "h" and exactly how frequent it was, etc., giving even better compression).

How about:

Huffman code works by laying out data in a tree. If you have a binary tree, you can associate every leaf to a code by saying that left child corresponds to a bit at 0 and right child to a 1. The path that leads from the root to a leaf corresponds to a code in a not ambiguous way.

This works for any tree and the prefix property is based on the fact that a leaf is terminal. Hence, you cannot go to leaf (have a code) by passing though another leaf (by having another code be a prefix).

The basic idea of Huffman coding is that you can build trees in such a way that the depth of every node is correlated with the probability of appearance of the node (codes more likely to happen will be closer the root).

There are several algorithms to build such a tree. For instance, assume you have a set of items you want to code, say a..f. You must know the probabilities of appearance every item, thanks to either a model of the source or an analysis of the actual values (for instance by analysing the file to code).

Then you can:

1. sort the items by probability
2. pickup the two items with the lowest probability
3. remove these items, group them in a new compound node and assign one item to left child (code 0) and the other to right child (code 1).
4. The probability of the compound node is the sum of individual probabilities and insert this new node in the sorted item list.
5. goto 2 while the number of items is >1

For the previous tree, it may correspond to a set of probabilities

a (0.5) b (0.2) c (0.1) d (0.05) e (0.05) f (0.1)

Then you pick items with the lowest probability (d and e), group them in a compound node (de) and get the new list

a (0.5) b (0.2) c (0.1) (de) (0.1) f (0.1)

And the successive item lists can be

a (0.5) b (0.2) c(de) (0.2) f (0.1)

a (0.5) b (0.2) (c(de))f (0.3)

a (0.5) b((c(de))f) (0.5)

a(b(((c(de))f)) 1.0

So the prefix property is insured by construction.