{"id":7094,"date":"2025-07-14T03:32:00","date_gmt":"2025-07-13T18:32:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=7094"},"modified":"2025-07-12T20:44:23","modified_gmt":"2025-07-12T11:44:23","slug":"flexolmo-open-language-models-for-flexible-data-use","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=7094","title":{"rendered":"FlexOlmo: Open Language Models for Flexible Data Use\u00a0"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>FlexOlmo: Open Language Models for Flexible Data Use\u00a0<\/strong>[184.9]<br>\u6211\u3005\u306f\u3001\u30c7\u30fc\u30bf\u5171\u6709\u306a\u3057\u3067\u5206\u6563\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3092\u30b5\u30dd\u30fc\u30c8\u3059\u308b\u65b0\u3057\u3044\u8a00\u8a9e\u30e2\u30c7\u30eb(LM)\u3067\u3042\u308bFlexOlmo\u3092\u7d39\u4ecb\u3057\u307e\u3059\u3002 FlexOlmo\u306f\u30a8\u30ad\u30b9\u30d1\u30fc\u30c8\u306e\u6df7\u6210\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3092\u63a1\u7528\u3057\u3066\u304a\u308a\u3001\u5404\u5c02\u9580\u5bb6\u306f\u30af\u30ed\u30fc\u30ba\u30c9\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067\u72ec\u7acb\u3057\u3066\u8a13\u7df4\u3055\u308c\u308b\u3002 \u6211\u3005\u306f\u3001\u516c\u958b\u30c7\u30fc\u30bf\u3067\u8a13\u7df4\u3055\u308c\u305f\u4e00\u822c\u5c02\u9580\u5bb6\u3068\u3001\u4ed6\u306e\u30c7\u30fc\u30bf\u6240\u6709\u8005\u304b\u3089\u72ec\u7acb\u3057\u305f\u8a13\u7df4\u3092\u53d7\u3051\u305f\u5c02\u9580\u5bb6\u3068\u3092\u52b9\u679c\u7684\u306b\u7d44\u307f\u5408\u308f\u305b\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u3053\u3068\u3092\u793a\u3059\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2507.07024v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2507.07024v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Wed, 09 Jul 2025 16:54:21 GMT)<\/li>\n\n\n\n<li>\u300cStandard MoEs train all experts and the router jointly on all data. In contrast, FLEXOLMO trains experts independently by teaching them to coordinate (\u00a73.3.1) and merges them at inference using a domain-informed router (\u00a73.3.2).\u300d\u3068\u9023\u5408\u5b66\u7fd2\u3084MoE\u3068\u805e\u3044\u3066\u601d\u3044\u6d6e\u304b\u3079\u308b\u304c\u73fe\u5b9f\u7684\u306b\u306f\u96e3\u3057\u3044\u305d\u308c\u305e\u308c\u306e\u5834\u6240\u3067\u69cb\u7bc9\u3055\u308c\u305fAI\u304c\u7d71\u5408\u7684\u306b\u52d5\u4f5c\u3059\u308b\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u306e\u63d0\u6848\u3068\u52b9\u679c\u691c\u8a3c\u3002<\/li>\n\n\n\n<li>\u300cOrganizations in regulated industries require LMs that can leverage their closed datasets while maintaining strict data privacy and access controls. Healthcare institutions, financial firms, and other entities possess valuable domain-specific data but cannot share it externally due to HIPAA, GDPR [14, 15], data sovereignty laws [16], and intellectual property (IP) protections. \u3000These organizations need training paradigms that enable AI improvement on their sensitive data while ensuring such sensitive data never leaves certain environments and can be removed from the model after training, e g , when data usage rights expire. In such settings, modular training approaches, where individual experts are trained independently and asynchronously on locally maintained data, are essential.\u300d\u306f\u307e\u3055\u306b\u305d\u306e\u901a\u308a\u3067\u975e\u5e38\u306b\u6709\u7528\u306a\u6280\u8853\u306b\u601d\u3048\u308b\u3002<\/li>\n\n\n\n<li>\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u30b5\u30a4\u30c8\u306f<a href=\"https:\/\/allenai.org\/blog\/flexolmo\">Introducing FlexOlmo: a new paradigm for language model training and data collaboration | Ai2<\/a>\u3001\u30ea\u30dd\u30b8\u30c8\u30ea\u306f<a href=\"https:\/\/github.com\/allenai\/FlexOlmo\">GitHub &#8211; allenai\/FlexOlmo: Code and training scripts for FlexOlmo<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[145,223,250],"class_list":["post-7094","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-federated-learning","tag-llm","tag-mixture-of-experts"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7094","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7094"}],"version-history":[{"count":1,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7094\/revisions"}],"predecessor-version":[{"id":7095,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7094\/revisions\/7095"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7094"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7094"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7094"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}