Topic:
Scalable Deep Learning Tutorial

A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learning paradigms, i.e. supervised, unsupervised, and reinforcement learning. Nevertheless, traditional deep learning approaches make use of cloud computing facilities and do not scale well to autonomous agents with low computational resources. Even in the cloud, they suffer from computational and memory limitations. They cannot be used to model adequately large physical worlds for agents which assume networks with billions of neurons. These issues are addressed in the last few years by the emerging topic of scalable deep learning which makes use of static and adaptive sparse connectivity in neural networks before and throughout training (or, on short, sparse training). The tutorial covers these research directions focusing on theoretical advancements, practical applications, and hands-on experience.

For the tutorial a GitHub repository is provided which you can find here: https://github.com/SelimaC/Tutorial-SCADS-Summer-School-2020-Scalable-Deep-Learning

Bio

Elena Mocanu is Assistant Professor in Machine Learning and Autonomous Agents at University of Twente, Netherlands. She received her B.Sc. degree in Mathematics and Physics from Transilvania University of Brasov, Romania, in 2004. After four years as a mathematics and physics teacher at the high-school level, Elena moved to the university. She has been an Assistant Lecturer within the Department of Information Technology, University of Bucharest, Romania from September 2008 to January 2011. In parallel, in 2009, she started a master program in Theoretical Physics. In 2011 she obtained the M.Sc. degree with a specialization in Quantum Transport from University of Bucharest, Romania. In January 2011, Elena moved from Romania to the Netherlands. She obtained the M.Sc. degree in Operations Research from Maastricht University, The Netherlands, in 2013. In her master thesis, she has investigated deep learning methods for “People detection for building automation” at NXP Semiconductors. In October 2013, Elena started her PhD research in Machine Learning and Smart Grids at TU Eindhoven. In January 2015 she performed a short research visit at the Technical University of Denmark and, from January to April 2016 she was a visiting researcher at University of Texas at Austin, USA. In 2017, Elena received her Doctor of Philosophy degree in Machine learning and Smart Grids from TU/e.

Decebal Mocanu is Assistant Professor in Artificial Intelligence and Machine Learning within the DMB group, Faculty of Electrical Engineering, Mathematics, and Computer Science at the University of Twente and Guest Assistant Professor within the Data Mining group, Department of Mathematics and Computer Science at the Eindhoven University of Technology. Since September 2017 until February 2020, Decebal was Assistant Professor in Artificial Intelligence and Machine Learning at TU Eindhoven and a member of TU Eindhoven Young Academy of Engineering. In 2017, he received his PhD in Artificial Intelligence and Network Science from TU Eindhoven. During his doctoral studies, Decebal undertook three research visits: the University of Pennsylvania (2014), Julius Maximilian University of Wurzburg (2015), and University of Texas, Austin (2016). Decebal medium-term research interest is to conceive scalable deep artificial neural network models and their corresponding learning algorithms using principles from network science, evolutionary computing, optimization and neuroscience. Such models shall have sparse and evolutionary connectivity, make use of previous knowledge, and have strong generalization capabilities to be able to learn, and to reason, using few examples in a continuous and adaptive manner. Inspired by Aristotle quote from Metaphysics “The whole is more than the sum of its parts”, in the long term, Decebal would like to study the synergy between artificial intelligence, neuroscience, and network science for the benefits of science and society.

Back to the Summer School 2020 overview