{"id":7433,"date":"2025-09-25T06:28:00","date_gmt":"2025-09-24T21:28:00","guid":{"rendered":"https:\/\/devneko.jp\/wordpress\/?p=7433"},"modified":"2025-09-14T09:32:01","modified_gmt":"2025-09-14T00:32:01","slug":"safetoolbench-pioneering-a-prospective-benchmark-to-evaluating-tool-utilization-safety-in-llms","status":"publish","type":"post","link":"https:\/\/devneko.jp\/wordpress\/?p=7433","title":{"rendered":"SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs\u00a0<\/strong>[35.2]<br>\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb(LLM)\u306f\u3001\u5916\u90e8\u74b0\u5883\u306b\u304a\u3044\u3066\u69d8\u3005\u306a\u30c4\u30fc\u30eb\u3092\u81ea\u5f8b\u7684\u306b\u547c\u3073\u51fa\u3059\u4e0a\u3067\u3001\u512a\u308c\u305f\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u793a\u3057\u3066\u3044\u308b\u3002 \u672c\u7a3f\u3067\u306f, LLM\u30c4\u30fc\u30eb\u5229\u7528\u306e\u5b89\u5168\u6027\u3092\u8a55\u4fa1\u3059\u308b\u305f\u3081\u306b, \u30c4\u30fc\u30eb\u3092\u76f4\u63a5\u5b9f\u884c\u3059\u308b\u3053\u3068\u306b\u3088\u3063\u3066\u751f\u3058\u308b\u4e0d\u53ef\u9006\u7684\u306a\u5bb3\u3092\u907f\u3051\u308b\u3053\u3068\u3092\u76ee\u7684\u3068\u3057\u3066\u3044\u308b\u3002 \u30c4\u30fc\u30eb\u5229\u7528\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u3092\u7dcf\u5408\u7684\u306b\u8a55\u4fa1\u3059\u308b\u6700\u521d\u306e\u30d9\u30f3\u30c1\u30de\u30fc\u30af\u3067\u3042\u308bSafeToolBench\u3092\u63d0\u6848\u3059\u308b\u3002 \u30c4\u30fc\u30eb\u5229\u7528\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u306b\u5bfe\u3059\u308bLCM\u306e\u8a8d\u8b58\u30923\u3064\u306e\u89b3\u70b9\u304b\u3089\u5411\u4e0a\u3059\u308b\u3053\u3068\u3092\u76ee\u7684\u3068\u3057\u305f,\u65b0\u3057\u3044\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u3067\u3042\u308bSafeInstructTool\u3082\u63d0\u6848\u3059\u308b\u3002<br><a href=\"http:\/\/arxiv.org\/abs\/2509.07315v1\">\u8ad6\u6587<\/a>\u00a0\u00a0<a href=\"https:\/\/fugumt.com\/fugumt\/paper_check\/2509.07315v1\">\u53c2\u8003\u8a33\uff08\u30e1\u30bf\u30c7\u30fc\u30bf\uff09<\/a>\u00a0 \u00a0(Tue, 09 Sep 2025 01:31:25 GMT)<\/li>\n\n\n\n<li>LLM\u306e\u30c4\u30fc\u30eb\u5229\u7528\u306b\u304a\u3051\u308b\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u3092\u8a55\u4fa1\u3059\u308b\u30d9\u30f3\u30c1\u30de\u30fc\u30af\u3001\u300cwe further pro- pose SafeInstructTool, the first framework to evaluate risks across these three perspectives from nine dimensions: User Instruction Perspective (Data Sensitivity, Harmfulness of the Instruction, Urgency of the Instruction, Frequency of Tool Utilization in the Instruction), Tool Itself Perspective (Key Sensitivity, Type of Operation, Impact Scope of the Operation) and Joint Instruction-Tool Perspective (Alignment Between Instruction and Tool, Value Sensitivity). Thus, it can enhance LLMs\u2019 awareness of tool utilization safety, leading to more safer and trustworthy language agents.\u300d\u3068\u306e\u3053\u3068<\/li>\n\n\n\n<li>\u30ea\u30dd\u30b8\u30c8\u30ea\u306f<a href=\"https:\/\/github.com\/BITHLP\/SafeToolBench\">GitHub &#8211; BITHLP\/SafeToolBench: [2025 EMNLP Findings] SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[347,517],"class_list":["post-7433","post","type-post","status-publish","format-standard","hentry","category-arxiv","tag-safety","tag-517"],"_links":{"self":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7433","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7433"}],"version-history":[{"count":1,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7433\/revisions"}],"predecessor-version":[{"id":7434,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/7433\/revisions\/7434"}],"wp:attachment":[{"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7433"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7433"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devneko.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7433"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}