Research

KAAI investigates what language models know, how this knowledge can be measured and represented, and how it can be connected with structured knowledge systems. We combine empirical analysis with methods and open research artifacts that make model knowledge inspectable, comparable and usable.

Research directions

Epistemic twins and language model knowledge

A central KAAI goal is to construct epistemic twins: discrete, comprehensive and comprehensible representations of the factual knowledge stored in language models. These representations enable systematic querying, comparison and scientific analysis of model knowledge.

Our flagship artifacts are GPTKB, which represents model knowledge as knowledge-base triples, and LLMpedia, which represents it as a human-readable encyclopedia.

Knowledge construction and consolidation

We develop methods to extract, integrate and consolidate knowledge from text, language models and heterogeneous sources. This includes entity disambiguation, relation extraction, knowledge-base completion and the assessment of conflicting claims.

Factuality, completeness and provenance

Reliable knowledge systems must make their limits visible. We study factuality, provenance, negative knowledge and the completeness of knowledge bases, developing methods that identify what a system knows, what it does not know, and how reliable its assertions are.

Commonsense and encyclopedic knowledge

We investigate broad, human-oriented forms of knowledge: commonsense, cultural knowledge and encyclopedic knowledge. Our work develops resources and methods for representing these forms of knowledge in ways that are both computationally useful and intelligible to people.

Research artifacts

• GPTKB — Queryable knowledge bases extracted from language models.
• LLMpedia — Wikipedia-style encyclopedias constructed from language-model knowledge.

Earlier systems and resources

• Ascent — Commonsense knowledge base construction.
• Mango — Cultural commonsense knowledge.
• Wikinegata — Negative knowledge.
• ReCoin — Relative completeness indicator for Wikidata.

Publications

For the complete and current publication record, see our Publications page.