Tree codebook and Binary Search:-

When a codebook is designed by the k-means algorithm, a quantisation of a vector during the design and the data transmission necessitates evaluating a distortion measure between the vector and each of the L reconstruction levels. This is called a full search and is responsible for the exponential dependence of the number of computation involved on the number of scalars in the vector and the number of bits per scalar.

Methods have been developed to eliminate this exponential dependence by modifying the codebook at the cost of higher average distortion and more storage. One such method is termed as a "tree-codebook". In tree codebook design method, assuming that L can be expressed as a power of 2, we first design a codebook with two reconstruction levels and using k-means algorithm. We then classify all the training vectors into two clusters, one cluster corresponding to and the other to . Each of the two clusters is treated independently, and a codebook with two reconstruction levels is designed for each cluster. The process is repeated until we have a total of L reconstruction levels at the last stage. This is shown in Figure (3.4.8 ) for the case L=8. By this process the tree codebook is designed.

Figure( 3.4.8)

 Computational and storage Requirements in designing the tree codebook.

  We again assume that one evaluation of the distortion measure requires N arithmetic operations. There are Log stages. The distortion measure is evaluated only twice for each of M training vectors for each stage and for each iteration of the K-means algorithm,

Total number of arithmetic operations/iteration =2NMlog₂L.

Compared to full-search method, the reduction in number of computations is by a factor or 2^NR/(2NR).

The storage required in the design of tree codebook is slightly more than the case of full search k-means algorithm, since the storage requirement is dominated in both cases by the need to store all the training vectors.

            We will now consider the computation involved in quantising a vector  using the tree codebook. In this procedure, we simply follow the tree and perform a search between two reconstruction levels at each node of the tree. Since the search is made between two levels a time, it is said to be a binary-search. Since log₂ L stages are involved and the distortion measure is evaluated at each stage, the number of arithmetic operations required to quantise using the tree codebook is given by In other words, the computational cost does not increase exponentially with N and R.

The reduction in computational cost for binary search, compared with the corresponding number for full search case is given by 2^NR/2NR when N=10 and R=2, the reduction is by a factor of 26,000. The binary search method discussed is a special case of a more general class of methods known as tree-searched vector quantisation. It is possible, for example to divide each node into more than two branches. In addition we can terminate a particular node earlier in the design of the codebook, when only a small number of training vectors are assigned to the node.