Holistic Optimization of Embedded Computer Vision Systems

Buckler, Mark Andrew

Holistic Optimization of Embedded Computer Vision Systems

Abstract

Despite strong interest in embedded computer vision, the computational demands of Convolutional Neural Network (CNN) inference far exceed the resources available in embedded devices. Thankfully, the typical embedded device has a number of desirable properties that can be leveraged to significantly reduce the time and energy required for CNN inference. This thesis presents three independent and synergistic methods for optimizing embedded computer vision: 1) Reducing the time and energy needed to capture and preprocess input images by optimizing the image capture pipeline for the needs of CNNs rather than humans. 2) Exploiting temporal redundancy within incoming video streams to perform computationally cheap motion estimation and compensation in lieu of full CNN inference for the majority of frames. 3) Leveraging the sparsity of CNN activations within the frequency domain to significantly reduce the number of operations needed for inference. Collectively these techniques significantly reduce the time and energy needed for computer vision at the edge, enabling a wide variety of exciting new applications.

Date Issued

2019-08-30

Keywords

Computer engineering; computer vision; Convolutional Neural Networks; Embedded Systems; Computer science; Imaging

Committee Chair

Sampson, Adrian L

Committee Member

Batten, Christopher
Hariharan, Bharath
De Sa, Christopher Matthew

Degree Discipline

Electrical and Computer Engineering

Degree Name

Ph.D., Electrical and Computer Engineering

Degree Level

Doctor of Philosophy

Types

dissertation or thesis