Position: Ph.D. Student
Current Institution: MIT
Energy-Efficient Circuits for Computational Photography on Mobile Devices
Computational photography encompasses a wide range of image capture and processing techniques, such as high dynamic range (HDR) imaging, low-light enhancement, image deblurring, panorama stitching and light field photography, that allow users to take photographs that can not be taken by a traditional digital camera. However, most of these techniques have high computational complexity, and existing software-based solutions do not achieve real-time performance and energy efficiency on mobile devices. This work proposes hardware accelerator-based implementations of these algorithms which achieve real-time performance. Additionally, the proposed implementations achieve over two orders of magnitude improvement in energy-efficiency making them suitable for integration into mobile devices.
The first part of this work focuses on deblurring of images degraded by camera shake blur. Removing this blur requires deconvolving the blurred image with a kernel which represents the trajectory of the camera during the exposure. This kernel is typically unknown and needs to be estimated from the blurred image. The estimation is computationally intensive and takes several minutes on a CPU which makes it unsuitable for mobile devices. This work presents the first hardware accelerator for kernel estimation for image deblurring applications. It achieves a 78x reduction in kernel estimation runtime, and a 56x reduction in total deblurring time for a FullHD 1920×1080 image, which enables quick feedback to the user. Configurability in kernel size and number of iterations gives up to 10x energy scalability, allowing the system to trade-off runtime with image quality. The test chip, fabricated in 40 nm CMOS, consumes 105 mJ for kernel estimation running at 83 MHz and 0.9 V, compared to 467J consumed by a CPU.
The second part of this work focuses on the design of a reconfigurable processor for bilateral filtering which is commonly used in computational photography applications. Specifically, the 40 nm CMOS test chip performs HDR imaging, low light enhancement and glare reduction while operating from 98 MHz at 0.9 V to 25 MHz at 0.9 V. It processes 13 megapixels per second while consuming just 17.8 mW at 98 MHz and 0.9 V, achieving significant energy reduction compared to previous CPU/GPU implementations.
These energy-scalable implementations pave the way for efficient integration of computationalphotography algorithms into mobile devices.
Priyanka Raina received a B.Tech. degree in Electrical Engineering from the Indian Institute of Technology (IIT) Delhi in 2011 and an S.M. degree in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology (MIT) in 2013. She is currently a Ph.D. candidate in the Energy-Efficient Circuits and Systems group at MIT, working under the supervision of Prof. Anantha Chandrakasan. She was a graduate research intern at Intel Labs in the summer of 2013, working on the design of a hardware accelerator for real time video enhancement using multi-frame super-resolution. Ms. Raina has received several awards and honors including the Institute Silver Medal for the highest GPA in Electrical Engineering at IIT Delhi and a Gold Medal at the Indian National Chemistry Olympiad. Her research interests include design of energy-efficient circuits for computational photography, computer vision and machine learning applications.