{"id":7425,"date":"2025-09-19T06:11:00","date_gmt":"2025-09-18T21:11:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=7425"},"modified":"2025-09-14T09:15:23","modified_gmt":"2025-09-14T00:15:23","slug":"steering-moe-llms-via-expert-deactivation","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=7425","title":{"rendered":"Steering MoE LLMs via Expert (De)Activation\u00a0"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>Steering MoE LLMs via Expert (De)Activation\u00a0<\/strong>[118.2]<br>LLM(Large Language Models)\u306b\u304a\u3051\u308bMixture-of-Experts (MoE)\u306f\u3001\u5404\u30c8\u30fc\u30af\u30f3\u3092\u5c02\u7528\u306eFeed-Forward Networks (FFN)\u306e\u30b5\u30d6\u30bb\u30c3\u30c8\u3092\u4ecb\u3057\u3066\u30eb\u30fc\u30c6\u30a3\u30f3\u30b0\u3059\u308b\u3002 \u6211\u3005\u306f,\u884c\u52d5\u95a2\u9023\u306e\u5c02\u9580\u5bb6\u3092\u691c\u77e5\u3057,\u5236\u5fa1\u3059\u308b\u3053\u3068\u3067,MoE\u30e2\u30c7\u30eb\u3092\u30b9\u30c6\u30a2\u30ea\u30f3\u30b0\u3059\u308b\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u3067\u3042\u308bSteerMoE\u3092\u63d0\u6848\u3059\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2509.09660v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2509.09660v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Thu, 11 Sep 2025 17:55:09 GMT)<\/li>\n\n\n\n<li>MoE\u3092\u64cd\u4f5c\u3057\u671b\u307e\u3057\u3044\uff08\u307e\u305f\u306f\u671b\u307e\u3057\u304f\u306a\u3044\uff09\u52d5\u4f5c\u306b\u8fd1\u3065\u3051\u308b\u624b\u6cd5\u306e\u63d0\u6848\u3002\u30cd\u30ac\u30c6\u30a3\u30d6\u306a\u65b9\u5411\u6027\u3067\u300cCritically, we are also exposing a novel dimension of \u201cAlignment Faking\u201d in LLMs (Greenblatt et al , 2024; Wang et al , 2024), where alignment is concentrated in a subset of experts, neglecting alternate routing paths that can catastrophically bypass alignment when triggered. We argue that, just as safety alignment must extend beyond the first few tokens (Qi et al , 2025), it must also go deeper than just a few expert pathways, ensuring robustness across the entire model routing topology.\u300d\u306f\u305d\u306e\u901a\u308a\u3060\u3068\u601d\u3046\u3002<\/li>\n\n\n\n<li>\u30ea\u30dd\u30b8\u30c8\u30ea\u306f<a href=\"https:\/\/github.com\/adobe-research\/SteerMoE\">GitHub &#8211; adobe-research\/SteerMoE: A framework for steering MoE models by detecting and controlling behavior-linked experts.<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[250],"class_list":["post-7425","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-mixture-of-experts"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7425","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7425"}],"version-history":[{"count":1,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7425\/revisions"}],"predecessor-version":[{"id":7426,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7425\/revisions\/7426"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7425"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}