From 12-13.12.2024, the workshop LLMs’ secrets – Learning Large Language Models for Knowledge Representation was held in Dresden. It has been organized as an exclusive and invitation-based event for outstanding AI researchers contributing to the field of Natural Language Processing (NLP) and Large Language Models. 13 participants joined the workshop, including speakers from TU Dresden, Karlsruhe Institute of Technology (KIT), and LMU Munich.

Program of the 1st Workshop of Learning Large Language Models for Knowledge Representation

Thursday, 12.12.2024

Keynotes

Michael Färber – LLMs and Knowledge Graphs for Science: From Papers to Insights

The rapid growth in scientific publications makes it increasingly difficult for researchers to stay up-to-date with the latest findings. Much of the valuable information — such as methods, datasets, and results — remains embedded in unstructured text, which limits its accessibility. In this presentation, Prof. Michael Färber explored how large language models (LLMs) and knowledge graphs can work together to extract key scientific insights and connect research across different fields. He also discussed how AI-driven recommendation systems, using these technologies, can help researchers find more relevant and transparent recommendations, making the research discovery process more efficient.

Ercong Nie – Probing Large Language Models: Multilingual Insights into Linguistic Form, Meaning, and Knowledge Representation

This talk delved into probing as a critical methodology for understanding large language models (LLMs), focusing on their multilingual capabilities and internal representations of linguistic form, meaning, and knowledge. Ercong Nie explored two probing paradigms: diagnostic probing, which examines internal neural representations, and prompting-based probing, which evaluates model outputs through behavioral tests. Key methods include minimal pair probing for isolating specific linguistic phenomena and calibrated prompting to address biases in multilingual datasets. Ercong Nie also introduced advanced techniques like decomposed prompting (CDPro) for sequence labeling tasks and multilingual probing for syntactic and semantic properties. Additionally, we discussed knowledge probing and editing, highlighting cross-lingual consistency and efficient methods for modifying LLM behavior while maintaining performance. By integrating insights from neurolinguistics and psycholinguistics, this talk provided a comprehensive overview of probing methodologies, offering valuable perspectives on how LLMs encode and process linguistic and knowledge-based information.

Ruotong Liao – Reasoning with Large Language Models (LLMs) from a temporal perspective, from temporal graphs and video understanding

Ruotong Liao introduced ECOLA, which enhances temporal knowledge graph embeddings using contextualized language representations. Next, she presented GenTKG, a retrieval-augmented generative framework leveraging LLMs for temporal knowledge forecasting with minimal training data, demonstrating superior cross-domain generalizability of LLMs in event forecasting tasks. She also introduced how we can utilize the temporal reasoning ability of foundation models in multimodal scenarios, e.g. analyzing events in Videos. Finally, Ruotong Liao discussed VideoINSTA, a zero-shot framework for understanding long videos through spatial-temporal reasoning with LLMs, setting new benchmarks in long-form video question-answering tasks. These works collectively highlight the transformative potential of LLMs for temporal and multimodal reasoning tasks, in connection with temporal data like graphs and videos.

Friday, 13.12.2024

Keynotes

Nicholas Popovič – Encoding or Decoding? NLU in the Age of Generative Language Models.

With the proliferation of generative language models and the continuing increase in their capabilities in natural language understanding tasks, decoder-only architectures have become more popular than encoder-only architectures. While in terms of raw benchmark performance, the results are often impressive, encoder-only language models retain inherent advantages in terms of efficiency and interpretability, especially for token classification tasks. In this talk, Nicholas Popovič explored areas in which both encoder-only and decoder-only architectures have their place and how to get the best of both worlds.

Chunlan Ma – Multilinguality in the Era of Large Language Models

This talk explored the opportunities and challenges of multilingualism in the era of large language models (LLMs) from three perspectives: model design, data availability, and cross-lingual transfer. Chunlan Ma analyzed how LLMs enable multilingual capabilities but also highlighted limitations in addressing low-resource languages. Specifically, we discussed advancements in multilingual pretraining, challenges in collecting diverse language data, and the potential of cross-lingual transfer to bridge resource gaps. The findings provided insights for advancing multilingual NLP in a rapidly evolving landscape.

Chair

The general chair of the workshop was Prof. Michael Färber, professor for Scalable Software Architectures for Data Analytics at ScaDS.AI Dresden/Leipzig. Shuzhou Yuan funcioned as a program chair. Tobias Schreieder, Parisa Aghdam, and Jin Zhang were the local chairs of the 1st Workshop of Learning Large Language Models for Knowledge Representation.

Prof. Dr.-Ing. Michael Färber

TUD Dresden University of Technology

Chair of Scalable Software Architectures for Data Analytics