Paul Franzon

North Carolina State University, USA

Memory Bandwidth Rich 3D Architectures for Machine Learning Training and Inference

Abstract

There exist multiple workloads that that benefit from high memory bandwidths but don’t need huge memory capacities nor low latency.   Such workloads include (1) training and inference of Recurrent Neural Networks (RNN), especially Long Short Term Memory (LSTM); (2) training and inference of Multi-Level Perceptron (MLP); and (3) inference of multiple parallel Deep Neural Networks (DNN).  At NC State, we have designed accelerators for each these based on leveraging the capabilities of the proposed Tezzaron DiRAM4 memory which we propose to replace with the 3DSoC technology.  A 2D version of one of these accelerators (LSTM/MLP) and a CNN engine is currently in fabrication.  The Tezzaron DiRAM4 memory is a 3D specific memory.  In its current proposed form the DiRAM4 supports up to 64 Gb of capacity at a rate of up to 8 Tb/s.  We re-architected the DiRAM4 for running multiple parallel DNNs directly from the DRAM.  The newly proposed DiRAM exposed 130 Tbps of memory bandwidth and used up to 45 Tbps of that bandwidth.  Thus all these designs show strong potential to highlight the capabilities the 3DIC technology if re-architected and re-implemented in that technology set.

Biography

Paul D. Franzon is currently the Cirrus Logic Distinguished Professor and the Director of Graduate programs in the Department of Electrical and Computer Engineering at North Carolina State University.  He earned his Ph.D. from the University of Adelaide, Adelaide, Australia.  He has also worked at AT&T Bell Laboratories, DSTO Australia, Australia Telecom, Rambus, and four companies he cofounded, Communica, LightSpin Technologies, Polymer Braille Inc. and Indago Technologies.  His current interests include applying machine learning to EDA, building AI accelerators, neuromorphic computing, RFID, advanced packaging, 2.5D and 3DICs and secure chip design. He has lead several major efforts and published over 300 papers in these areas.  In 1993 he received an NSF Young Investigators Award, in 2001 was selected to join the NCSU Academy of Outstanding Teachers, in 2003, selected as a Distinguished Alumni Professor, received the Alcoa Research Award in 2005, and the Board of Governors Teaching Award in 2014.  He served with the Australian Army Reserve for 13 years as an Infantry Soldier and Officer.  He is a pilot in the Bandit Flight Team, a formation flying unit. He is a Fellow of the IEEE.