Training Language Models to Explain Their Own Computations
Training Language Models to Explain Their Own Computations [73.9] 本研究では,LMの自己内部への特権的アクセスをどの程度活用できるかを考察し,その振る舞いを説明するための新しい手法を提案する。 既存の解釈可能性技術を用いて,(1)LM特徴によって符号化された情報,(2)LMの内部アクティベーションの因果構造,(3)特定の入力トークンがLM出力に与える影響の自然言語記述を生成する。 論文参考訳（メタデータ） (Wed, 12 Nov 2025 02:05:44 GMT)
「Taken together, these results suggest that even when language models cannot faithfully self-explain as a result of ordinary training, they can learn to do so through an objective that enforces consistency between their external explanations and their internal procedures. This reframes interpretation as not only an external analysis problem, but as a capability that can be trained into LMs themeselves; by leveraging privileged access to internal computations, “introspective interpretability” techniques offer an avenue towards scalable understanding of model behavior.」と非常に興味深い研究