(Disclaimer: Let us remember, Independent Component Analysis (ICA) may not be achievable in general since (1) there may be no independent components, and (2) you might make fatal errors in estimating the component distributions. We only call it ICA because everyone else does.)

Explanation: ICA is about factoring probability distributions, and doing blind source separation. It is related to lots of other things - entropy and information maximisation, maximum likelihood density estimation (MLE), EM (expectation maximisation, which is MLE with hidden variables)and projection pursuit. It is basically a way of finding special linear (non-orthogonal) co-ordinate systems in multivariate data, using higher-order statistics in various ways. If you don't understand, read the papers below!

Applications: anywhere you have ensembles of multivariate data, eg: anywhere you might use PCA (Principal Components Analysis). Examples include blind separation (eg: of mixed speech signals), biomedical data processing (eg: of EEG [brainwave] data), finding `features' in data (eg: learning edge-detectors for ensembles of natural images).

Algorithms: There are many different algorithms, but often they are not so different really. Read on.

-Tony Bell


A selection of papers that I have access to right now. There is no pretence to completeness, so sorry if you're omitted! Email me if you have something you'd like to include. Many other papers are collected at Paris Smaragdis's site listed below.

Quick guide to the papers:
Our main work is in B1 and B3: infomax/ICA for source separation and image coding respectively. The related natural gradient approach (which we now use) is in A1, a special case of the algorithm proposed in CU. The relations between these algorithms and maximum likelihood (C1, PH) are gone through in C2, MK, PP and O1 (so we'd better believe it!). The infomax origins can be traced in NP. A nice review from the Finnish school appears in KA. D1 is a book containing much material on ICA. Clever extensions and analyses are appearing in G1, PN, T1, T2, and finally PP, where temporal context gives better separation. Perhaps the best work on source separation/deconvolution is in LA (a pun). Our work on EEGs and ERPs is in M1 and M2.

[NOTE: All papers are PostScript file. Compressed versions are X-compressed. They have ".ps.Z" on the end, and need UNIX `uncompress' to make them ".ps" files.]

The following two papers are probably under-cited, containing, respectively, `early' theoretical and practical insights into algorithms which are part of the infomax/maximum-likelihood/natural-gradient family of ICA algorithms. Unfortunately we don't have them online. Does anyone else? Also, let us not forget the French originators: Herault/Jutten, Comon and others.


Basic ICA code in MATLAB (as used in Bell and Sejnowski 1996) It's simple.

Scott Makeig et al's MATLAB 4.2 routines for applying ICA to psychophysiological data (tar-file)


Demo of real-room blind separation/deconvolution of two sources


Hastily assembled selection of Web pages with more ICA papers and details.
Paris Smaragdis' page has a huge amount of material. It is more-or-less a
superset of this page.

Paris Smaragdis' ICA & BSS page (MIT) GO TO THIS PAGE. IT'S GREAT.

Allan Barros's ICA page in Japan. Another huge collection of stuff.

Jean-François Cardoso

Andrzej Cichocki

Erkki Oja

Scott Makeig,

Shun-ichi Amari

J.-P. Nadal : Publications

Mark Girolami's Home Page

Homepage of Aapo Hyvärinen

NIPS*96 Signal Processing Workshop
(several new papers).

The WWW homepage of Juha Karhunen

BSS Research (Experiments at RIKEN in Japan)



11/96 -Tony Bell (tony@salk.edu), Computational Neurobiology Lab , Salk Institute, San Diego.
back to T.Bell Home Page