{"id":4505,"date":"2024-03-04T06:17:00","date_gmt":"2024-03-03T21:17:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=4505"},"modified":"2024-03-04T06:17:00","modified_gmt":"2024-03-03T21:17:00","slug":"language-specific-neurons-the-key-to-multilingual-capabilities-in-large-language-models","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=4505","title":{"rendered":"Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models&nbsp;<\/strong>[122.3]<br>\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb(LLM)\u306f\u3001\u7279\u5225\u306b\u30ad\u30e5\u30ec\u30fc\u30c8\u3055\u308c\u305f\u591a\u8a00\u8a9e\u4e26\u5217\u30b3\u30fc\u30d1\u30b9\u3067\u4e8b\u524d\u8a13\u7df4\u3055\u308c\u308b\u3053\u3068\u306a\u304f\u3001\u9855\u8457\u306a\u591a\u8a00\u8a9e\u6a5f\u80fd\u3092\u793a\u3059\u3002 LLM\u5185\u306e\u8a00\u8a9e\u7279\u7570\u7684\u30cb\u30e5\u30fc\u30ed\u30f3\u3092\u8b58\u5225\u3059\u308b\u305f\u3081\u306e\u65b0\u3057\u3044\u691c\u51fa\u624b\u6cd5\u3067\u3042\u308b\u8a00\u8a9e\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u78ba\u7387\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc(LAPE)\u3092\u63d0\u6848\u3059\u308b\u3002 \u4ee5\u4e0a\u306e\u7d50\u679c\u304b\u3089,LLM\u304c\u7279\u5b9a\u306e\u8a00\u8a9e\u3092\u51e6\u7406\u3067\u304d\u308b\u80fd\u529b\u306f,\u795e\u7d4c\u7d30\u80de\u306e\u30b5\u30d6\u30bb\u30c3\u30c8\u304c\u5c11\u306a\u3059\u304e\u308b\u305f\u3081\u3067\u3042\u308b\u3053\u3068\u304c\u793a\u5506\u3055\u308c\u305f\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2402.16438v1\">\u8ad6\u6587<\/a>&nbsp;&nbsp;<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2402.16438v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>&nbsp; &nbsp;(Mon, 26 Feb 2024 09:36:05 GMT)<\/li>\n\n\n\n<li>LLM\u306e\u591a\u8a00\u8a9e\u5bfe\u5fdc\u304c\u6d45\u3044\u5c64\u306e\u6bd4\u8f03\u7684\u5c11\u6570\u306e\u30cb\u30e5\u30fc\u30ed\u30f3\u306b\u3088\u3063\u3066\u5b9f\u73fe\u3055\u308c\u3066\u3044\u308b\u306e\u3067\u306f\u306a\u3044\u304b\uff1f\u3068\u3044\u3046\u5831\u544a\u3002LAPE: Language Activation Probability Entropy\u3068\u3044\u3046\u6307\u6a19\u3092\u4f5c\u308a\u30011\u3064\u304b2\u3064\u306e\u8a00\u8a9e\u306b\u306e\u307f\u5f37\u304f\u53cd\u5fdc\u3059\u308b\u30cb\u30e5\u30fc\u30ed\u30f3\u3092\u7279\u5b9a\u3057\u3066\u3044\u308b\u3088\u3046\u3002<\/li>\n\n\n\n<li>mBERT\u306e\u6642\u4ee3\u304b\u3089\u610f\u5916\u3068\u5bb9\u6613\u306b\u591a\u8a00\u8a9e\u6027\u304c\u5f97\u3089\u308c\u3066\u3044\u305f\u306e\u3067\u7d0d\u5f97\u611f\u306e\u3042\u308b\u7d50\u679c\u3002LoRA\u306a\u3069\u3067\u591a\u8a00\u8a9e\u6027\u304c\u7834\u58ca\u3055\u308c\u306a\u3044\u3088\u3046\u306b\u898b\u3048\u308b\u306e\u3082\u540c\u3058\u7406\u7531\u306a\u3093\u3060\u308d\u3046\u304b\u3002\u8ad6\u6587\u306b\u3082\u3042\u308b\u901a\u308a\u9078\u629e\u7684\u306b\u591a\u8a00\u8a9e\u7279\u6027\u3092\u6b8b\u305b\u308b\u3068\u9762\u767d\u3044\u3068\u601d\u3046\u3002<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>How do Large Language Models Handle Multilingualism?\u00a0<\/strong>[87.1]<br>\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb(LLM)\u306f\u3001\u69d8\u3005\u306a\u8a00\u8a9e\u3067\u9855\u8457\u306a\u6027\u80fd\u3092\u793a\u3059\u3002 LLM\u306e\u591a\u8a00\u8a9e\u5165\u529b\u51e6\u7406\u3092\u8a18\u8ff0\u3057\u305f\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u3092\u63d0\u6848\u3059\u308b\u3002 \u3055\u3089\u306b,\u7279\u5b9a\u306e\u8a00\u8a9e\u51e6\u7406\u306b\u304a\u3051\u308b\u8a00\u8a9e\u7279\u7570\u7684\u30cb\u30e5\u30fc\u30ed\u30f3\u306e\u5b58\u5728\u306b\u3064\u3044\u3066\u691c\u8a0e\u3059\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2402.18815v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2402.18815v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Thu, 29 Feb 2024 02:55:26 GMT)<\/li>\n\n\n\n<li>\u5225\u30c1\u30fc\u30e0\u306b\u3088\u308b\u5831\u544a\u3060\u304c\u691c\u8a3c\u3057\u3066\u3044\u308b\u300cwe introduce a hypothesis suggesting that LLMs address multilingualism by first translating queries into English, processing them using English with the help of multilingual knowledge, and then translating the responses back into the original language.\u300d\u306f\u4e0a\u8a18\u306b\u8fd1\u3044\u3088\u3046\u306b\u601d\u3046\u3002<\/li>\n\n\n\n<li>\u300cMoreover, enhancing the multilingual capabilities of LLMs can be achieved by fine-tuning languagespecific neurons with merely 200 contextual examples.\u300d\u3082\u885d\u6483\u7684\u3002<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[223,267],"class_list":["post-4505","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-llm","tag-multilingual"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/4505","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4505"}],"version-history":[{"count":0,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/4505\/revisions"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4505"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4505"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4505"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}