Huffman Coding : (continued)

On one hand, the actual bit rate may be considerably higher than average bit rate. In the opposite situation, the actual bit rate may be lower than average bit rate. To transmit message with variable length codewords, through a fixed bit rate system, a buffer that can store the message when the actual bit rate increases must be maintained. Maintaining a buffer imposes additional complexity on the coding system and involves delay.

If we are allowed to collect as many message as we can with and then assign a codeword jointly, we can design codewords with an average bit rate/message arbitrarily close to entropy H. Huffman coding is also called an entropy coding method. Consider the case when

If we assign a codeword to one message at a time, the average bit rate is 1 bit/message .Suppose we wait until we have 2 messages and then assign jointly codewords as:

Joint message	Probability	Huffman code
	1/64	000
	7/64	001
	7/64	01
	49/64	1

The average bit rate is bits/message

As we increase the number of messages that are coded jointly, the average bit rate obtained by Huffman coding approaches the entropy. In practice we cannot wait for a large number of messages since this involves delay, and would require a large additional codebook.Therefore average bit rate achieved by Huffman coding is higher than entropy.