{"id":6994,"date":"2025-06-30T05:41:00","date_gmt":"2025-06-29T20:41:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=6994"},"modified":"2025-06-28T20:55:14","modified_gmt":"2025-06-28T11:55:14","slug":"the-ideation-execution-gap-execution-outcomes-of-llm-generated-versus-human-research-ideas","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=6994","title":{"rendered":"The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas \/ Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI\u00a0"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas&nbsp;<\/strong>[90.3]<br>\u826f\u3044\u30a2\u30a4\u30c7\u30a2\u306f\u5358\u306b\u65ac\u65b0\u306a\u3082\u306e\u3067\u306f\u306a\u304f\u3001\u5b9f\u884c\u5f8c\u306b\u3088\u308a\u826f\u3044\u7814\u7a76\u304c\u3082\u305f\u3089\u3055\u308c\u308b\u3079\u304d\u3067\u3042\u308b\u3002 AI\u304c\u751f\u307f\u51fa\u3059\u30a2\u30a4\u30c7\u30a2\u304c\u3088\u308a\u826f\u3044\u7814\u7a76\u6210\u679c\u3092\u3082\u305f\u3089\u3059\u304b\u3069\u3046\u304b\u3092\u30c6\u30b9\u30c8\u3059\u308b\u305f\u3081\u306b\u3001\u6211\u3005\u306f\u5b9f\u884c\u7814\u7a76\u3092\u884c\u3046\u3002 \u5b9f\u884c\u524d\u5f8c\u306e\u540c\u3058\u30a2\u30a4\u30c7\u30a2\u306e\u30ec\u30d3\u30e5\u30fc\u30b9\u30b3\u30a2\u3092\u6bd4\u8f03\u3059\u308b\u3068\u3001LLM\u751f\u6210\u306e\u30a2\u30a4\u30c7\u30a2\u306e\u30b9\u30b3\u30a2\u306f\u5c02\u9580\u5bb6\u306b\u3088\u308b\u30a2\u30a4\u30c7\u30a2\u3088\u308a\u3082\u5927\u5e45\u306b\u6e1b\u5c11\u3059\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2506.20803v1\">\u8ad6\u6587<\/a>&nbsp;&nbsp;<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2506.20803v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>&nbsp; &nbsp;(Wed, 25 Jun 2025 19:47:23 GMT)<\/li>\n\n\n\n<li>LLM\u304c\u51fa\u3057\u305f\u30a2\u30a4\u30c7\u30a2\u3068\u5c02\u9580\u5bb6\u306e\u30a2\u30a4\u30c7\u30a2\u3092\u300cOur execution participants spend an average of 103 hours executing the assigned idea and then submit the codebase and paper to document their experiments. All projects are then reviewed blindly by our recruited expert reviewers\u300d\u3068\u8a55\u4fa1\u3057\u305f\u3068\u3053\u308d\u300cAverage scores of AI ideas drop significantly more than Human ideas in the execution study across all the evaluation metrics.\u300d\u3068\u3044\u3046\u6307\u6458\u3002<\/li>\n\n\n\n<li>\u3084\u306f\u308a\u4eba\u9593\u306e\u5c02\u9580\u5bb6\u306f\u6df1\u304f\u8003\u3048\u3066\u3044\u308b\u3088\u3046\u3068\u3044\u3046\u8208\u5473\u6df1\u3044\u7d50\u679c\u3002\u540c\u6642\u306b\u3001\u30a2\u30a4\u30c7\u30a2\u306e\u307f\u3060\u3068AI\u306e\u8a55\u4fa1\u304c\u9ad8\u3044\u3068\u3044\u3046\u3053\u3068\u306f\u30a2\u30a4\u30c7\u30a2\u3060\u3057\u3067\u306f\u6709\u52b9\u306a\u306e\u3067\u306f\u306a\u3044\u304b\uff1f\u3068\u304b\u6700\u7d42\u7684\u306a\u30b9\u30b3\u30a2\u3067\u3082\u305d\u3053\u305d\u3053\u5065\u95d8\u3057\u3066\u3044\u308b\u306e\u3067\u306f\u306a\u3044\u304b\uff1f\u3068\u898b\u3048\u306a\u304f\u3082\u306a\u3044\u3002\u4e0b\u8a18\u8ad6\u6587\u306e\u3088\u3046\u306bAI\u79d1\u5b66\u8005\u306e\u5b9f\u73fe\u53ef\u80fd\u6027\u306f\u9ad8\u307e\u3063\u3066\u3044\u308b\u3088\u3046\u306b\u601d\u3046\u3002<\/li>\n\n\n\n<li>\u30ea\u30dd\u30b8\u30c8\u30ea\u306f<a href=\"https:\/\/github.com\/NoviScl\/AI-Researcher\">GitHub &#8211; NoviScl\/AI-Researcher<\/a><\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI\u00a0<\/strong>[98.2]<br>\u77e5\u7684\u79d1\u5b66\u7814\u7a76\u6240(ISL)\u306e\u30d1\u30e9\u30c0\u30a4\u30e0\u3092\u63d0\u6848\u3059\u308b\u3002 ISL\u306f\u3001\u8a8d\u77e5\u3068\u5177\u4f53\u7684\u77e5\u6027\u3092\u6df1\u304f\u7d71\u5408\u3057\u305f\u591a\u5c64\u30af\u30ed\u30fc\u30ba\u30c9\u30eb\u30fc\u30d7\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u3067\u3042\u308b\u3002 \u3053\u306e\u3088\u3046\u306a\u30b7\u30b9\u30c6\u30e0\u306f\u3001\u73fe\u5728\u306e\u79d1\u5b66\u7684\u767a\u898b\u306e\u9650\u754c\u3092\u514b\u670d\u3059\u308b\u305f\u3081\u306b\u4e0d\u53ef\u6b20\u3067\u3042\u308b\u3001\u3068\u6211\u3005\u306f\u4e3b\u5f35\u3059\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2506.19613v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2506.19613v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Tue, 24 Jun 2025 13:31:44 GMT)<\/li>\n\n\n\n<li>\u300c1) Foundation Models provide multi-modal scientific knowledge representation and closed-loop learning capabilities, supporting complex reasoning and domain adaptation; (2) Agent Layer dynamically orchestrates scientific workflows\u2014including hypothesis generation, literature review, experimental planning, execution, and analysis\u2014while integrating model\/toolkit via MCP integration; (3) Embodied Layer realizes robust physical interaction through advanced perception, navigation, and manipulation modules, enabling precise, adaptive operations in real-world laboratory environments.\u300d\u304b\u3089\u306a\u308bAI\u79d1\u5b66\u8005\u30fbAI\u30e9\u30dc\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u306e\u63d0\u6848\u3002<\/li>\n\n\n\n<li>\u73fe\u72b6\u3068\u8ab2\u984c\u304c\u3068\u3066\u3082\u53c2\u8003\u306b\u306a\u308b\u3002<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[42,125,623],"class_list":["post-6994","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-autonomous-agent","tag-embodied","tag-623"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/6994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6994"}],"version-history":[{"count":2,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/6994\/revisions"}],"predecessor-version":[{"id":6997,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/6994\/revisions\/6997"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6994"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}