High-Performance Vision System Exploiting Key Features of Visual Cortex

Technology #11985

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Professor Tomaso Poggio
Department of Brain and Cognitive Sciences, MIT
External Link (poggio-lab.mit.edu)
Professor Lior Wolf
Department of Computer Science, Tel Aviv University
External Link (www.cs.tau.ac.il)
Professor Thomas Serre
Department of CLPS, Brown University
External Link (www.brown.edu)
Professor Maximilian Riesenhuber
Department of Neuroscience, Georgetown University
External Link (maxlab.neuro.georgetown.edu)
Stanley Bileschi
Department of Brain and Cognitive Sciences, MIT
Managed By
Daniel Dardani
MIT Technology Licensing Officer
Patent Protection

High-performance vision system exploiting key features of visual cortex

US Patent 7,606,777
Object recognition with features inspired by visual cortex
Computer Vision and Pattern Recognition, 2005


The Inventors have developed a high-performance method for object recognition that is modeled from the human visual cortex. Object detection and recognition may be used for a broad set of tasks such as image classification, face detection and vehicle detection. It is applicable to a rapidly growing number of industries including health sciences, manufacturing and security.

Problem Addressed  

Object recognition typically involves the computation of a set of target features at one step and their combination in successive steps. Features usually fall in one of two categories: template-based or histogram-based. While these methods have success, they struggle with object transformation and generic object recognition, respectively. Inspired by the hierarchical nature of the primate visual cortex, the Inventors introduce a new set of biologically-inspired features that exhibit a better trade-off between invariance and selectivity than existing methods for most holistic, robust object recognition.


In this approach, features are obtained by combining the response of local edge-detectors over neighboring positions and multiple orientations, mimicking complex cells in the primary visual cortex. These features are both flexible, allowing for small distortions of the input, and selective, preserving local feature geometry. For an input image, a set of features learned from a training set are computed. A standard classifier is then run on this feature vector. The resulting approach is able to learn from very few examples while being simpler than existing recognition methods.

The system follows the standard model of object recognition in the primate cortex, in a feedforward hierarchy for visual processing. In its simplest version, the standard model consists of four layers of computation units in which simple S units, which increase object selectivity, alternate with complex C units, which introduce gradual invariance to scale and translation. The model has been able to quantitatively duplicate the generalization properties exhibited by neurons in the inferotemporal cortex that remain highly selective for particular objects while being invariant to ranges of scales and positions. The Inventors extend the standard model to learn a vocabulary of visual features from natural images. This model can robustly recognize many object categories and is competitive with state-of-the-art object recognition systems.


  • System is highly successful in recognizing both generic and transforming objects
  • Model invariant to range of feature position and scales
  • Simple approach can learn from fewer labeled examples yet shown to outperform most existing object recognition systems