An information-maximisation approach to blind separation and
Anthony J. Bell & Terrence J. Sejnowski
We derive a new self-organising
learning algorithm which maximises the information transferred
in a network of non-linear units. The algorithm does not assume any knowledge
of the input distributions, and is defined here for the zero-noise limit.
Under these conditions, information maximisation has extra properties
not found in the linear case (Linsker 1989). The non-linearities in the
transfer function are able to pick up higher-order moments of the input
distributions and perform something akin to true redundancy reduction between
units in the output representation. This enables the network to separate
statistically independent components in the inputs: a higher-order
generalisation of Principal Components Analysis.
We apply the network to the source separation (or cocktail party) problem,
successfully separating unknown mixtures of up to ten speakers. We also
show that a variant on the network architecture is able to perform
blind deconvolution (cancellation of unknown echoes and reverberation in a
speech signal). Finally, we derive dependencies of information transfer
on time delays. We suggest that information maximisation provides a unifying
framework for problems in `blind' signal processing.
07/95 -Tony Bell (email@example.com).
back to Home Page