Generative AI models are trained on large collections of existing texts, images, music, and other creative works. In some cases, they reproduce passages that closely resemble material from their training data. But does such an output demonstrate that a work has been memorized by the model? Or can the reproduction instead be explained by information contained in the prompt, common language patterns, external retrieval, or chance?

This question is highly relevant to current copyright disputes. One prominent example is GEMA vs. OpenAI. In November 2025, the Regional Court of Munich I considered claims concerning the reproduction of protected song lyrics in ChatGPT outputs and their alleged memorization within the underlying language models. The court largely upheld GEMA’s claims. The case raises technical questions that remain unresolved: What does it mean for a work to be “stored” in a neural model? What can actually be inferred from the reproduction of a text? And how reliable is such reproduction as evidence of memorization?

In this thesis, you will investigate these questions from a Computer Science and NLP perspective. The aim is to develop and experimentally evaluate methods for determining under which conditions model outputs provide reliable evidence of training-data memorization. The thesis will be supervised at TU Dresden and ScaDS.AI in collaboration with Julia Möller-Klapperich, who will contribute the copyright-law perspective.

What are the tasks?

Analyse the technical questions behind GEMA vs. OpenAI

• Study the technical assumptions made in the case and in the surrounding legal and scientific discussion.
• Distinguish between training-data membership, memorization, extraction, regurgitation, and textual similarity.
• Determine which technical claims can — and cannot — be supported by observing model outputs.
• Translate the relevant questions into precise and experimentally testable hypotheses.

The thesis is not intended to provide an independent legal assessment of the judgment. Instead, the case serves as a motivating example and a real-world test case for the technical investigation.

Design controlled experiments

• Work with open language models and legally suitable material, such as public-domain, licensed, or synthetic texts.
• Create controlled examples for which it is known whether, when, and how often a text occurred during training or continued pre-training.
• Test whether trained passages can be reproduced using different prompting strategies and decoding settings.
• Compare the results with matched texts that were not included in the training data.
• Control for factors such as textual predictability, prompt–target overlap, and the occurrence of common phrases.

Evaluate methods for detecting memorization

• Implement and compare existing approaches based on text completion, repeated prompting, token or n-gram overlap, edit distance, semantic similarity, model likelihoods, or membership-inference methods.
• Investigate false positives and false negatives produced by these methods.
• Analyse whether successful reproduction is caused by actual memorization or largely by cues already contained in the prompt.
• Depending on the final direction, develop an improved auditing method or evaluation protocol that provides more reliable evidence of memorization.

Possible research questions include:

• Does successful reproduction reliably indicate that a text was part of the training data?
• How much information may a prompt reveal before a reproduction test becomes uninformative?
• Which metrics best distinguish memorization from coincidental, predictable, or prompt-induced similarity?
• How do model size, training frequency, training stage, and decoding strategy affect reproduction?
• Can instruction tuning or output filters suppress the reproduction of a work while information about it remains present in the model?
• Can evidence be aggregated across several passages to make more reliable statements about an entire work?

The precise scope and research questions will be selected together based on the student’s interests and the available models and data.

Expected outcome

The thesis should result in a reproducible and methodologically rigorous study of memorization and text reproduction in generative models. A strong outcome could include:

• a controlled benchmark for evaluating memorization tests;
• an improved auditing or evaluation method;
• a systematic analysis of the reliability of existing approaches;
• or new empirical findings concerning the relationship between training exposure, memorization, and reproduction.

The goal is not merely to collect individual examples of reproduced text, but to derive generalizable conclusions about what such examples reveal about a model.

What prerequisites do you need?

• Good foundations in machine learning and natural language processing.
• Experience with PyTorch, Hugging Face Transformers, or Large Language Models is highly desirable.
• Interest in careful experimental design, quantitative evaluation, and interdisciplinary research.
• Very good English. German is helpful for engaging with the German court case and legal literature.
• Ambition to conduct a rigorous, high-quality thesis that could form the basis of a scientific publication.

Why this thesis is special

• You will investigate an open, highly timely technical problem with direct societal, legal, and economic relevance.
• You will receive interdisciplinary supervision combining expertise in AI/NLP and copyright law.