Huffman Coding :

One optimal codeword design method that is simple to use and which is uniquely decodable and which result in lowest average bit rate is Huffman coding.

An example of Huffman coding is shown in Figure (4.12). In the example L = 6 with the probability for each message possibility noted at each node.

Message	Codeword	Probability
	0
	100
	110
	1110
	101
	1111

In the I^st step of Huffman coding, we select the two message possibilities that have two lowest probabilities. We combine them and form a new node with combined probabilities. We assign '0' to one of the two branches and '1' to other. Reversing this affects the codeword but not the average bit rate. We continue with this process until we are left with one message with probability '1'. To determine the specific codeword assigned to each message possibility, we begin with last node with probability '1', follow the branches that lead to the message possibility of interest and combine the 0's and 1's on the branches.

For example, has codeword 1110. To compare performance of Huffman coding with the entropy H and uniform length codeword assignment for the above example, we compute average bit rate achieved by uniform length codeword, Huffman coding and the entropy respectively.

For uniform length codeword: 3 bit/message

Huffman coding:

Entropy:

In the example chosen, the Huffman coding results in an average bit rate that is close to the entropy and that is lower than uniform length codeword by more than 1 bit/message. The potential reduction in the average bit rate when VLC assignment is used comes with cost. The actinal bit rate is variable,when message possibilities with low probabilities are coded. The codeword obtained are shown in figureabove. This implies, messege possibilities with higher probabilities has shorter codewords and message with low probabilities longer codewords. This is referred to as variable length coding (VLC).

(continued in the next slide)