The junior research group “Automated Machine Learning” wants to develop machine learning approaches that can configure themselves automatically by learning from past data. Consequently, the research shall contribute to fulfilling the promise of artificial intelligence and lead to complete end-to-end learning systems.

Therefore, this junior research group is focusing on following key topics:

• Automated machine learning
• Neural architecture search
• Bayesian hyperparameter optimization
• Resource efficient large language models

Projects

Currently, the research group ‘Automated Machine Learning’ is working on the project “Compressing Large Language Models via Neural Architecture Search”. Large language models mark a new era in artificial intelligence. However, their large size poses challenges in deploying them for inference in
real-world applications, due to significant GPU memory requirements and high inference latency. This
project explores neural architecture search to find sub-parts of the fine-tuned network that optimally trade-off efficiency, for example in terms of model size or latency, and generalization performance

Additionally, the group will be exploiting scaling laws to automated large language model training in future projects.

Team

Lead

The junior research group leader, Aaron Klein, started at ScaDS.AI Leipzig in July 1st 2024. At the University of Freiburg, Klein studied Computer Science for his Master’s, and started his PhD in Machine Learning.

Previously, Klein worked as an Applied Scientist at Amazon Web Services. His research area focuses on Automated Machine Learning and Large Language Models. So far, Klein’s scientific achievements include

• Autosklearn – automated machine learning framework on top of sklearn,
• the development of efficient methods for model-based hyperparameter optimization,
• his work on neural architecture search,
• and the Co-Organization on several workshops and the AutoML Seminar.

More information about him, his publications, and his work can be found on his personal website.

Team Members

Publications

By now, no publications of the group exist.