Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering
Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering [61.1] 本研究では,中間層におけるルーティング動作を,支配言語に関連する共有専門家に適応的に誘導するルーティング誘導型ステアリング手法を提案する。 論文参考訳（メタデータ） (Tue, 20 Jan 2026 15:04:25 GMT)
「anguages within the same linguistic family tend to share similar routing distributions, whereas linguistically distant languages are routed through more distinct subsets of experts (cf. Section 4.2). Moreover, both routing similarity and expert utilization display a pronounced layerwise structure.」、「Dominant languages serve as central hubs for cross-lingual capacity sharing, high-resource languages rely heavily on shared experts, whereas low-resource languages depend more on language- exclusive experts yet remain weak」と納得感があり、かつ、興味深い結果