Independent Component Analysis viewed as a Tensor Decomposition
The problem of identifying linear mixtures of independent random variables only from outputs can be traced back to 1953 with the works of Darmois or Skitovich. They pointed out that when data are non Gaussian, a lot more can be said about the mixture. In practice, Blind Identification of linear mixtures is useful especially in Factor Analysis, in addition to many other application areas (including Signal & Image Processing, Digital Communications, Biomedical, or Complexity Theory). Harshman and Carroll provided independently in the seventies numerical algorithms to decompose a data record stored in a 3-way array into elementary arrays, each representing the contribution of a single underlying factor. The main difference with the well known Principal Component Analysis is that the mixture is not imposed to be a unitary matrix. This is very relevant because the actual mixture often has no reason to have orthogonal columns. The Parafac ALS algorithm, widely used since that time, theoretically does not converge for topological reasons, but yields very usable results after a finite number of iterations, under rank conditions however. Independently, the problem of Blind Source Separation (BSS) arose around 1985 and was solved -explicitly or implicitly- with the help of High-Order Statistics (HOS), which are actually tensors. It gave rapidly birth to the more general problem of Independent Component Anlalysis (ICA) in 1991. ICA is a tool that can be used to extract Factors when the physical diversity does not allow to store directly and efficiently the data in tensor format, in other words when the data cannot be uniquely decomposed directly. The problem then consists of decomposing a data cumulant tensor, of arbitrarily chosen order, into a linear combination of rank-one symmetric tensors. For this purpose, several algorithms can be thought of.