Practical Applications of Mutual Information: Phylogenetic Trees, Independent Component Analysis, and Network Inferences

The mutual information (MI) between X an Y is the amount by which 
the information (either in Shannon or Kolmogorove sense) needed 
to specify one is decreased when the other is known. It is 
non-negative and symmetric, and vanishes only when X and Y are 
independent, therefore it is a very natural candidate for a 
universal similarity measure. I will first review the general 
concepts and practical methods for estimating MI. The I will 
present three applications: (a) Independent component analysis, 
illustrated by the problem of disentangling the EEG of a 
pregnant woman into the one of the mother and the one of the 
fetus; (b) Classifying "texts", either of natural languages 
or of DNA. In the latter case, I will discuss MI-based
methods to construct phylogenetic trees from mitochondrial 
DNA; and (c) Attempts to reconstruct gene regulation networks
from gene expression data obtained by microarrays.