This course presents an algorithm-defined full-stack approach to accelerate emerging workloads that are set to enable machine intelligence in the next generation autonomous system. The course will first introduce BitFusion, our most recent acceleration paradigm for deep neural networks that builds upon the algorithmic insight that bitwidth of operations in DNNs can be reduced without compromising their classification accuracy. However, to prevent loss of accuracy, the bitwidth varies significantly across DNNs and it may even be adjusted for each layer individually. Thus, a fixed-bitwidth accelerator would either offer limited benefits to accommodate the worst-case bitwidth requirements, or inevitably lead to a degradation in final accuracy. To alleviate these deficiencies, this work introduces dynamic bit-level fusion/decomposition as a new dimension in the design of DNN accelerators. We explore this dimension by designing Bit Fusion, a bit-flexible accelerator that constitutes an array of bit-level processing elements that dynamically fuse to match the bitwidth of individual DNN layers. This flexibility in the architecture enables minimizing the computation and the communication at the finest granularity possible with no loss in accuracy. Utilizing algorithmic insights enables us to match the server-grade GPU performance for DNN acceleration within milli-Watt regime. This architecture is the foundation of the second version of DnnWeaver, the very first open-source acceleration stack for deep learning (www.dnnweaver.org). I will extend the discussion to how, very recently, we have been able to automate the process of deep quantizing neural networks (below 8 bits) using deep reinforcement learning.
Dr. Esmaeilzadeh was awarded early tenure at the University of California, San Diego (UCSD), where he is the inaugural holder of the Halicioglu Chair in Computer Architecture with the rank of associate professor in Computer Science and Engineering. Prior to UCSD, he was an assistant professor in the School of Computer Science at the Georgia Institute of Technology from 2013 to 2017. There, he was the inaugural holder of the Catherine M. and James E. Allchin Early Career Professorship. Hadi is the founding director of the Alternative Computing Technologies (ACT) Lab, where his team is developing new technologies and cross-stack solutions to build the next generation computer systems. He is also the associate director of Center for Machine Integrated Computing and Security (MICS) at UCSD. Dr. Esmaeilzadeh obtained his Ph.D. from the Department of Computer Science and Engineering at the University of Washington in 2013 where his Ph.D. work received the 2013 William Chan Memorial Best Dissertation Award. Prof. Esmaeilzadeh received the IEEE Technical Committee on Computer Architecture (TCCA) Young Architect Award in 2018 and was inducted to the ISCA Hall of Fame in the same year. He has received the Air Force Office of Scientific Research Young Investigator Award (2017), College of Computing Outstanding Junior Faculty Research Award (2017), Qualcomm Research Award (2017 and 2016), Google Research Faculty Award (2016 and 2014), Microsoft Research Award (2017 and 2016), and Lockheed Inspirational Young Faculty Award (2016). His teams were awarded the Qualcomm Innovation Fellowship in 2014 and 2018, one of his students was a Microsoft Research Fellow, and another won the 2017 National Center for Women & IT (NCWIT) Collegiate Award. Four of his undergraduate students have been awarded the Georgia Tech Presidents Undergraduate Research Award (PURA). His research has been recognized by four Communications of the ACM Research Highlights, four IEEE Micro Top Picks, a nomination for Communications of the ACM Research Highlights, an honourable mention in IEEE Micro Top Picks, and a Distinguished Paper Award in HPCA 2016. Hadis work on dark silicon has also been profiled in New York Times. More information is available on his webpage, http://cseweb.ucsd.edu/~hadi/.