Machine learning and neuroscience discover the mathematical system used by the brain to organize visual objects
June 5, 2020
Slices of brain with specific regions highlighted, suspended over images of faces

The brain represents visual objects through four networks tuned for four quadrants of object space: faces, bodies, spiky objects, and stubby objects. The same object space can be discovered using a deep network. Credit: Olivier Wyart

When Plato set out to define what made a human a human, he settled on two primary characteristics: We do not have feathers, and we are bipedal (walking upright on two legs). Plato's characterization may not encompass all of what identifies a human, but his reduction of an object to its fundamental characteristics provides an example of a technique known as principal component analysis.

Now, Caltech researchers have combined tools from machine learning and neuroscience to discover that the brain uses a mathematical system to organize visual objects according to their principal components. The work shows that the brain contains a two-dimensional map of cells representing different objects. The location of each cell in this map is determined by the principal components (or features) of its preferred objects; for example, cells that respond to round, curvy objects like faces and apples are grouped together, while cells that respond to spiky objects like helicopters or chairs form another group.