Believe It or Not: How Deeply do LLMs Believe Implanted Facts?

MATS 学员:

Clément Dumas, Julian Minder, Rowan Wang

作者:

Stewart Slocum, Julian Minder, Clément Dumas, Henry Sleight, Ryan Greenblatt, Samuel Marks, Rowan Wang

引用次数

12 Citations

摘要:

Knowledge editing techniques promise to implant new factual knowledge into large language models (LLMs). But do LLMs really believe these facts? We develop a framework to measure belief depth and use it to evaluate the success of knowledge editing techniques. We operationalize belief depth as the extent to which implanted knowledge 1) generalizes to related contexts (e.g. Fermi estimates several logical steps removed), 2) is robust to self-scrutiny and direct challenge, and 3) is represented similarly to genuine knowledge (as measured by linear probes). Our evaluations show that simple prompting and mechanistic editing techniques fail to implant knowledge deeply. In contrast, Synthetic Document Finetuning (SDF) - where models are trained on LLM-generated documents consistent with a fact - often succeeds at implanting beliefs that behave similarly to genuine knowledge. However, SDF's success is not universal, as implanted beliefs that contradict basic world knowledge are brittle and representationally distinct from genuine knowledge. Overall, our work introduces measurable criteria for belief depth and enables the rigorous evaluation necessary for deploying knowledge editing in real-world applications.

近期研究

非大国冲突与 AI 风险

作者:

Kristina Kempkey

日期:

八月 26, 2026

引用次数:

合成角色预训练:从第一个 token 开始对齐

作者:

Julian Minder

日期:

八月 13, 2026

引用次数:

常见问题

什么是 MATS 项目?
项目持续多久?