SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems

Xiaofan Zhang, Haoming Lu, Cong Hao, Jiachen Li, Bowen Cheng, Yuhong Li, Kyle Rupnow, Jinjun Xiong, Thomas Huang, Honghui Shi, Wen-mei Hwu, Deming Chen

University of Illinois at Urbana-Champaign, Inspirit IoT, Inc., IBM Research


Object detection and tracking are challenging tasks for resource-constrained embedded systems. While these tasks are among the most compute-intensive tasks from the artificial intelligence domain, they are only allowed to use limited computation and memory resources on embedded devices. In the meanwhile, such resource-constrained implementations are often required to satisfy additional demanding requirements such as real-time response, high-throughput performance, and reliable inference accuracy. To overcome these challenges, we propose SkyNet, a hardware-efficient neural network to deliver the state-of-the-art detection accuracy and speed for embedded systems. Instead of following the common top-down flow for compact DNN design, SkyNet provides a bottom-up DNN design approach with comprehensive understanding of the hardware constraints at the very beginning to deliver hardware-efficient DNNs. SkyNet is demonstrated by winning the extremely competitive System Design Contest for low power object detection in the 56th IEEE/ACM Design Automation Conference (DAC-SDC), where our SkyNet significantly outperforms all other 100+ competitors. SkyNet is also extended to GOT-10K for generic object tracking. For the state-of-the-art object trackers SiamRPN++ and SiamMask, where ResNet-50 is employed as the backbone, implementations using our SkyNet as the backbone are faster with better or similar accuracy and smaller in terms of parameter size.


SkyNet is demonstrated on the DAC-SDC official test set by delivering 0.731 Intersection over Union (IoU) and 67.33 frames per second (FPS) using a TX2 GPU;  and 0.716 IoU and 25.05 FPS using an Ultra96 FPGA. SkyNet can also handle object tracking with 1.60X~1.73X higher FPS, and 32.20X smaller parameter size compared to a ResNet-50 backbone adopted by the state-of-the-art Siamese trackers.

  • Results for object detection

  • Results for object tracking


Demo 1

Small object detection for real-life UAV applications using DAC-SDC dataset.

Demo 2

Live demo at IEEE/ACM Design Automation Conference, 2019. The proposed SkyNet is running on a TX2 GPU and an Ultra96 FPGA, respectively, for real-time object detection.


Invited Talks about SkyNet

Conference / Seminar
Google PhD Fellowship Summit
Virtual Event
Jul. 2020


If you find SkyNet useful, please cite the SkyNet paper: