# Candid Covariance-Free Incremental Principal Component Analysis

J. Weng, Y. Zhang, W. Hwang

Published 2003 · Computer Science

Appearance-based image analysis techniques require fast computation of principal components of high-dimensional image vectors. We introduce a fast incremental principal component analysis (IPCA) algorithm, called candid covariance-free IPCA (CCIPCA), used to compute the principal components of a sequence of samples incrementally without estimating the covariance matrix (so covariance-free). The new method is motivated by the concept of statistical efficiency (the estimate has the smallest variance given the observed data). To do this, it keeps the scale of observations and computes the mean of observations incrementally, which is an efficient estimate for some well known distributions (e.g., Gaussian), although the highest possible efficiency is not guaranteed in our case because of unknown sample distribution. The method is for real-time applications and, thus, it does not allow iterations. It converges very fast for high-dimensional image vectors. Some links between IPCA and the development of the cerebral cortex are also discussed.

This paper references

10.1201/9780429499661

Introduction to the theory of neural computation

J. Hertz (1991)

Proceedings of NSF/DARPA Workshop on Development and Learning

(2000)

10.1016/0022-247X(85)90131-3

On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix

E. Oja (1985)

10.1007/978-3-642-77643-4

Motion and Structure from Image Sequences

J. Weng (1993)

and R

J. Weng (2000)

Adaptive spectral analysis technique for unbiased frequency estimation in the presence of white noise

P. Thompson (1979)

10.1016/0893-6080(89)90044-0

Optimal unsupervised learning in a single-layer linear feedforward neural network

T. Sanger (1989)

Convergence Analysis of Complementary Candid Incremental Principal Component Analysis ∗

Y. Zhang (2001)

Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network,

T. D. Sanger (1989)

10.1162/jocn.1991.3.1.71

Eigenfaces for Recognition

M. Turk (1991)

10.1016/0165-1684(84)90028-8

Subspace methods of pattern recognition

E. Oja (1983)

vanLoan, Matrix Computations

C.F.G.H. Golub (1989)

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib

10.1364/JOSAA.4.000519

Low-dimensional procedure for the characterization of human faces.

L. Sirovich (1987)

10.1007/978-3-662-46221-8_16

Probability Theory and Mathematical Statistics

I. Bronshteĭn (1987)

10.1109/CVPR.1997.609311

The FERET evaluation methodology for face-recognition algorithms

P. J. Phillips (1997)

10.2307/3621013

Matrix Computations

G. Golub (1983)

10.1109/72.883430

State-based SHOSLIF for indoor visual navigation

S. Chen (2000)

10.1006/cviu.2000.0837

Appearance-Based Hand Sign Recognition from Intensity Image Sequences

Y. Cui (2000)

10.1007/BF00198094

Development of feature detectors by self-organization

J. Rubner (1990)

Developmental Humanoids: Humanoids that Develop Skills Automatically

J. Weng (2000)

Numerical recipes in C

W. Press (2002)

10.1007/BF01421486

Visual learning and recognition of 3-d objects from appearance

H. Murase (1995)

10.1001/jama.1986.03370210189033

Principles of Neural Science

M. Alexander (1986)

Advanced Engineering Mathematics, Maple Computer Guide

E. Kreyszig (2000)

10.1109/ICASSP.1978.1170425

Adaptive data orthogonalization

N. Owsley (1978)

This paper is referenced by

10.1109/ICAPR.2009.39

Online Improved Eigen Tracking

Subarna Tripathi (2009)

LONGITUDINAL HIGH-DIMENSIONAL DATA ANALYSIS

Vadim Zipunnikov (2011)

10.1016/J.AASRI.2013.10.052

An Efficient Distributed Anomaly Detection Model for Wireless Sensor Networks

Murad A. Rassam (2013)

10.1109/CRV.2013.47

Eigenbackground Bootstrapping

Kevin Hughes (2013)

Slowness learning for curiosity-driven agents

Varun Raj Kompella (2014)

10.1155/2014/429451

EVD Dualdating Based Online Subspace Learning

Bo Jin (2014)

Online Multi-Stage Deep Architectures for Feature Extraction and Object Recognition

Derek C. Rose (2013)

Learning error-correcting representations for multi-class problems

Bautista Martín (2016)

10.1142/S0218001408006545

Face Recognition by Artificial Vision Systems: a Cognitive Perspective

B. Raducanu (2008)

10.1111/INSR.12220

Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?

H. Cardot (2015)

TheMultilayer In-Place Learning NetworkfortheDevelopment of GeneralInvariances andMulti-Task Learning

X. Xue (2007)

10.1109/ICPR.2006.231

An Adaptive Classification Algorithm Using Robust Incremental Clustering

Herward Prehn (2006)

10.1016/j.neunet.2016.10.001

An online incremental orthogonal component analysis method for dimensionality reduction

Tao Zhu (2017)

An Acceleration Scheme for Mini-batch, Streaming PCA

Salaheddin Alakkari (2019)

10.1007/s00500-019-04330-7

Infinite Lattice Learner: an ensemble for incremental learning

Justin Lovinger (2020)

10.1109/ICACC.2010.5486889

Incremental Principal Component Analysis Based On Reduced Subspace Projection

Cao Xiang-hai (2010)

10.1109/TAMD.2010.2072150

Top–Down Connections in Self-Organizing Hebbian Networks: Topographic Class Grouping

Matthew D. Luciw (2010)

10.1007/978-3-642-30232-9_12

Review of the Application of Matrix Information Theory in Video Surveillance

M. K. Bhuyan (2013)

10.1002/ECJ.10430

An improvement of incremental recursive fisher linear discriminant for online feature extraction

Ryohei Ohta (2013)

10.3745/KTSDE.2016.5.8.361

Dimension Reduction Methods on High Dimensional Streaming Data with Concept Drift

Cheong Hee Park (2016)

10.1007/978-3-030-28553-1_2

Algorithms for Optimization and Machine Learning over Cloud

Ratnik Gandhi (2020)

10.1007/11760023_91

Object Detection Using Unit-Linking PCNN Image Icons

Xiaodong Gu (2006)

10.1109/ISCAS.2007.378086

Incremetal Spatio-Temporal Feature Extraction and Retrieval for Large Video Database

B. Geng (2007)

Muscle Vectors as Temporally Dense “Labels”

X. Wu (2020)

10.1109/TAMD.2009.2021698

Dually Optimal Neuronal Layers: Lobe Component Analysis

J. Weng (2009)

10.1109/CVPR.2012.6248056

Foreground detection using spatiotemporal projection kernels

Y. Moshe (2012)

10.1007/s11063-009-9117-1

A New Incremental PCA Algorithm With Application to Visual Learning and Recognition

Dong Huang (2009)

10.1007/11564386_24

Real-Time Modeling of Face Deformation for 3D Head Pose Estimation

Kenji Oka (2005)

The Fast Convergence of Incremental PCA

A. Balsubramani (2013)

10.1109/GlobalSIP45357.2019.8969411

Stochastic Principal Component Analysis Via Mean Absolute Projection Maximization

Mayur Dhanaraj (2019)

10.3724/SP.J.1001.2009.03391

Efficient Kernel Principal Component Analysis Algorithm for Large-Scale Data Set: Efficient Kernel Principal Component Analysis Algorithm for Large-Scale Data Set

Weiya Shi (2009)

10.1016/j.neucom.2015.11.012

Orthogonal component analysis: A fast dimensionality reduction algorithm

Tao Zhu (2016)

See more