{"id":7206,"date":"2025-07-29T05:29:00","date_gmt":"2025-07-28T20:29:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=7206"},"modified":"2025-07-27T11:32:22","modified_gmt":"2025-07-27T02:32:22","slug":"diffusion-beats-autoregressive-in-data-constrained-settings","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=7206","title":{"rendered":"Diffusion Beats Autoregressive in Data-Constrained Settings\u00a0"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>Diffusion Beats Autoregressive in Data-Constrained Settings\u00a0<\/strong>[46.1]<br>\u81ea\u5df1\u56de\u5e30(AR)\u30e2\u30c7\u30eb\u306f\u9577\u3044\u9593\u3001\u5927\u304d\u306a\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30e9\u30f3\u30c9\u30b9\u30b1\u30fc\u30d7\u3092\u652f\u914d\u3057\u3066\u304d\u305f\u3002 \u8fd1\u5e74,AR\u30e2\u30c7\u30eb\u3088\u308a\u3082\u30a2\u30c9\u30d0\u30f3\u30c6\u30fc\u30b8\u304c\u4f4e\u3044\u3082\u306e\u306e,\u62e1\u6563\u578b\u8a00\u8a9e\u30e2\u30c7\u30eb\u304c\u5c06\u6765\u6027\u306e\u3042\u308b\u9078\u629e\u80a2\u3068\u3057\u3066\u6d6e\u4e0a\u3057\u3066\u3044\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2507.15857v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2507.15857v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Mon, 21 Jul 2025 17:59:57 GMT)<\/li>\n\n\n\n<li>\u300cIn this paper, we systematically study masked diffusion models in data-constrained settings\u2014where training involves repeated passes over limited data\u2014and find that they significantly outperform AR models when compute is abundant but data is scarce. Diffusion models make better use of repeated data, achieving lower validation loss and superior down- stream performance.\u300d\u3068\u3044\u3046\u6307\u6458\u3002\u76f4\u89b3\u7684\u306b\u3082\u305d\u3046\u3060\u308d\u3046\u3068\u601d\u3046\u3002<\/li>\n\n\n\n<li>\u30ea\u30dd\u30b8\u30c8\u30ea\u306f<a href=\"https:\/\/diffusion-scaling.github.io\/\">Diffusion Beats Autoregressive in Data-Constrained Settings<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[114],"class_list":["post-7206","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-diffusion-model"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7206","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7206"}],"version-history":[{"count":1,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7206\/revisions"}],"predecessor-version":[{"id":7207,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7206\/revisions\/7207"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7206"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7206"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7206"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}