LLMs consistently default to Western-centric norms, entities, and assumptions when reasoning about everyday situations, even when prompted in other languages — recent benchmarking work has documented this gap directly, for instance Naous et al.’s analysis of how models balance Arab and Western representations across stereotype generation, fairness, and cultural entity prediction tasks in Arabic prompts. Separately, a growing body of evidence suggests that cultural bias is not fixed by simply translating benchmarks into more languages, since reasoning, behavior, and communication are themselves shaped by culture — i.e., multilingual competence and cultural alignment are distinct properties that don’t automatically travel together. This thesis asks whether that gap can be localized mechanistically inside a model and then closed with a lightweight, interpretable post-training intervention, rather than full-scale retraining. arxivEmergent Mind

What are the tasks?

Understand the cultural-bias and interpretability landscape

• Survey existing cultural-awareness benchmarks and their scope/format — e.g. CANDLE, which collects cultural commonsense assertions from web corpora, and GEOMLAMA, which assesses cultural commonsense via masked sentences in multilingual settings — alongside newer resources such as BLEnD, NormAd, CulturalBench, Camellia, and DRISHTIKON. arxiv
• Review mechanistic-interpretability tooling applicable here: probing classifiers, causal tracing / activation patching, and sparse autoencoders for isolating human-interpretable directions.
• Review existing post-training approaches to cultural alignment (e.g., CARE’s multilingual preference learning, CulFiT’s fine-grained cultural training) as baselines to contrast against.

Localize culture-encoding representations

• Use causal mediation analysis / activation patching on cultural-QA prompts (e.g., items from BLEnD or NormAd) to test whether “defaulting to a dominant culture” is driven by identifiable layers, attention heads, or a low-rank direction in activation space.
• Test cross-lingual overlap: is the same internal signal responsible for cultural defaulting across languages (a shared, language-agnostic “culture direction”), or is it language-specific — directly probing the multilingual-vs-multicultural gap noted above.

Design a lightweight post-training intervention

• Derive steering vectors or a minimal targeted edit (contrastive activation addition, small-scale representation editing, or a lightweight LoRA adapter) from the localized signal.
• Apply the intervention to steer outputs toward culturally appropriate responses without full RLHF/DPO-scale retraining.
• Compare cost, effectiveness, and interpretability against a full preference-optimization baseline.

Multilingual and cross-cultural generalization

• Test whether an intervention derived in one language/culture pair transfers zero-shot to others.
• Include at least one lower-resource case study (e.g., an Indic language, alongside DRISHTIKON-style Indian-culture items) in addition to high-resource languages, to probe whether generalization holds outside data-rich settings.
• Run regression checks to confirm the intervention doesn’t degrade general capability or factuality — since prior surveys note LLMs already show systematic performance gaps favoring cultures overrepresented in pretraining data, and sharp accuracy drops under cross-lingual or code-switched cultural reasoning, any fix needs to be checked for side effects on both fronts. https://arxiv.org/pdf/2505.19484

Evaluation & analysis

• Quantitative evaluation on held-out cultural-QA items (BLEnD/NormAd/CulturalBench-style).
• Qualitative/human evaluation with culturally-informed judgment where feasible.
• Interpretability report visualizing the identified culture-related circuit(s)/direction(s) and how the intervention shifts them.

Prerequisites

• Strong interest in multilingual NLP, cultural bias, and mechanistic interpretability.
• Good Python skills; experience with transformer internals (hooks, activation extraction) or libraries like TransformerLens is a plus.
• Comfort designing controlled probing/ablation experiments.
• Very good English for reading and writing.

Why this thesis is special

• Sits at a genuinely active but underexplored intersection: most cultural-bias work is benchmark-only or full-scale fine-tuning; interpretability-guided, lightweight steering for cultural (as opposed to toxicity/safety) alignment is largely unexplored.
• Directly tests the emerging “multilingual ≠ multicultural” finding at the mechanistic level rather than just the behavioral/benchmark level.
• Strong publication potential given the fast-moving cultural-NLP benchmark literature (BLEnD, NormAd, CulturalBench, Camellia, DRISHTIKON all published within the last two years) actively looking for alignment methods, not just more evaluation.
• Reusable pipeline: the localization + lightweight steering method could extend beyond culture to other bias/safety dimensions.