Figure 6   Signal estimation problem

Source and Channel Coding

           One of the major areas of application of probability theory is Information theory and coding. In 1948 Claude Shannon published the paper "A mathematical theory of communication"   which lays the foundation of modern digital communication. Following are two remarkable results stated in simple languages :

• Digital data is efficiently represented with number of bits for a symbol decided by its probability of occurrence.
                                   
• The data at a rate smaller than the channel capacity can be transmitted over a noisy channel with arbitrarily small probability of error. The channel capacity again is determined from the probabilistic descriptions of the signal and the noise.