Description |
The mutual information (MI) between X an Y is the amount by which
the information (either in Shannon or Kolmogorove sense) needed
to specify one is decreased when the other is known. It is
non-negative and symmetric, and vanishes only when X and Y are
independent, therefore it is a very natural candidate for a
universal similarity measure. I will first review the general
concepts and practical methods for estimating MI. The I will
present three applications: (a) Independent component analysis,
illustrated by the problem of disentangling the EEG of a
pregnant woman into the one of the mother and the one of the
fetus; (b) Classifying "texts", either of natural languages
or of DNA. In the latter case, I will discuss MI-based
methods to construct phylogenetic trees from mitochondrial
DNA; and (c) Attempts to reconstruct gene regulation networks
from gene expression data obtained by microarrays.
|