On the 03-February-2022 at 11:00 a.m. the seventh lecture of the Living Lab lecture series took place. In this talk, ScaDS.AI scientific researcher Jan Frenzel talked about Big Data Performance Analysis. Add this event to your calendar (iCal).
Big Data Performance Analysis
Jan Frenzel introduced the audience to the area of performance evaluation and performance investigation of these frameworks. Furthermore, Jan Frenzel presented benefits of using an established performance analysis tool, Vampir, as an alternative to the dashboards of Apache Spark and Apache Flink.
In the last years, the amount of data that needs to processed has increased tremendously. Java-based frameworks, such as Apache Hadoop, Apache Spark and Apache Flink have been developed to simplify the work with distributed data by hiding much of the complexity related to distributed data processing, such as splitting data or moving data in the compute cluster, behind functional building blocks. However, because of this hidden complexity, performance analysis of applications written with these frameworks is particularly challenging. The performance could be limited by the application, the framework itself or the framework’s configuration. Different approaches could be used to investigate these potential causes of low performance. |
Missed this lecture on Big Data Performance Analysis?
You can rewatch this lecture on YouTube.
Mit dem Laden des Videos akzeptieren Sie die Datenschutzerklärung von YouTube.
Mehr erfahren
Video laden
Living Lab Lecture Series
The Living Lab Lecture Series gives you an in-depth insight into the many research topics of ScaDS.AI Dresden/Leipzig. From Natural Language Processing to Ethics and Moral Code in AI, a great variety of topics are discussed. You can join our lectures every first thursday of the month or watch them on YouTube afterwards. If you have ideas for topics to discuss in the future, please let our Living Lab team know. We suggest for you to regularly check our event calendar, to never miss out on upcoming lectures or other interesting events organized by or in cooperation with our center.
Find out more about our Living Lab Lecture Series.