{"id":136104,"date":"2025-09-09T02:00:14","date_gmt":"2025-09-09T02:00:14","guid":{"rendered":"http:\/\/cryptospotters.net\/?p=136104"},"modified":"2025-09-09T02:00:14","modified_gmt":"2025-09-09T02:00:14","slug":"popular-ai-model-performance-benchmark-may-be-flawed-meta-researchers-warn","status":"publish","type":"post","link":"http:\/\/cryptospotters.net\/?p=136104","title":{"rendered":"Popular AI model performance benchmark may be flawed, Meta researchers warn"},"content":{"rendered":"<p>Source: Tech &#8211; South China Morning PostA popular benchmark for measuring the performance of artificial intelligence models could be flawed, a group of Meta Platforms researchers warned, raising fresh questions on the veracity of evaluations that have been made on major AI systems.<br \/>\n\u201cWe\u2019ve identified multiple loopholes with SWE-bench Verified,\u201d wrote Jacob Kahn, manager at Meta AI research lab Fair, in a post last week on the developer platform GitHub.<br \/>\nThe post from Fair, which stands for Fundamental AI Research, found several&#8230;<a href=\"https:\/\/www.scmp.com\/tech\/tech-trends\/article\/3324735\/popular-ai-model-performance-benchmark-may-be-flawed-meta-researchers-warn?utm_source=rss_feed\" target=\"_blank\" class=\"feedzy-rss-link-icon\" rel=\"noopener\">Read More<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>Source: Tech &#8211; South China Morning PostA popular benchmark for measuring the performance of artificial intelligence models could be flawed, a group of Meta Platforms researchers warned, raising fresh questions&hellip; <\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"_links":{"self":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts\/136104"}],"collection":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=136104"}],"version-history":[{"count":0,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts\/136104\/revisions"}],"wp:attachment":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=136104"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=136104"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=136104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}