These Ph.D. students from the Georgia Tech School of Electrical and Computer Engineering (ECE) were honored for their paper entitled “NeuroSensor: A 3D Image Sensor with Integrated Neural Accelerator.” Their co-authors on the paper are their Ph.D. advisor and ECE Professor Saibal Mukhopadhyay, ECE Regents’ Professor Sudhakar Yalamanchili, and Denny Lie, Mukhopadhyay’s former Ph.D. student who now works at Intel. Amir, Kim, and Kung are all current members of the Gigascale Reliable Energy Efficient Nanosystem (GREEN) Lab, which is led by Mukhopadhyay.

Neuro-inspired deep learning algorithms have emerged as a key enabler for performing complex tasks such as feature extraction, classification, and recognition on image/video data. Currently, such complex processing is mainly performed at the back-end machines. There is a growing need to push learning algorithms into the sensor front-end to facilitate fast decision making and reduce the demand on communication bandwidth. However, the size, weight, and power constraints limit the ability to perform complex processing in the sensor.

This paper presents an innovative system design concept to enable integration of deep learning architecture within a image sensor. The device is known as the NeuroSensor – a 3D stacked image sensor with an integrated neural network. The NeuroSensor performs on-chip neural computation for in-field decision making and classification, and thus reduces latency and energy consumption for data movement between the sensor and host. The design and analysis presented in the paper creates a new research direction on how to enable complex machine learning at the sensor front-end.

This paper is a key milestone in the research performed at the GREEN lab on intelligent and autonomous image sensors. This is also one of many key publications, and the second best paper award received by GREEN Lab students in 2016 on this research topic. The research is supported in part by Mukhopadhyay’s National Science Foundation CAREER Award and Office of Naval Research Young Investigator Award.