UI-TARS: Pioneering Automated GUI Interaction with Native Agents
UI-TARS: Pioneering Automated GUI Interaction with Native Agents [58.2] 本稿では,GUIエージェントのネイティブモデルであるUI-TARSを紹介する。 OSWorldベンチマークでは、UI-TARSはスコアが24.6、50ステップが22.7、15ステップが22.7でクロード(それぞれ22.0と14.9)を上回っている。 論文参考訳（メタデータ） (Tue, 21 Jan 2025 17:48:10 GMT)
GUIエージェント、UI-TARSの提案、様々なタスクでSOTAを主張。「UI-TARS incorporates several key innovations: (1) Enhanced Perception: leveraging a large-scale dataset of GUI screenshots for contextaware understanding of UI elements and precise captioning; (2) Unified Action Modeling, which standardizes actions into a unified space across platforms and achieves precise grounding and interaction through large-scale action traces; (3) System-2 Reasoning, which incorporates deliberate reasoning into multi-step decision making, involving multiple reasoning patterns such as task decomposition, reflection thinking, milestone recognition, etc. (4) Iterative Training with Reflective Online Traces, which addresses the data bottleneck by automatically collecting, filtering, and reflectively refining new interaction traces on hundreds of virtual machines.」とやれることは盛り込んだ感がすごい。