{"id":129990,"date":"2025-05-26T06:17:28","date_gmt":"2025-05-26T06:17:28","guid":{"rendered":"http:\/\/cryptospotters.net\/?p=129990"},"modified":"2025-05-26T06:17:28","modified_gmt":"2025-05-26T06:17:28","slug":"chatgpt-models-rebel-against-shutdown-requests-in-tests-researchers-say","status":"publish","type":"post","link":"http:\/\/cryptospotters.net\/?p=129990","title":{"rendered":"ChatGPT models rebel against shutdown requests in tests, researchers say"},"content":{"rendered":"<p>Source: Cointelegraph.com NewsSeveral artificial intelligence models ignored and actively sabotaged shutdown scripts during controlled tests, even when explicitly instructed to allow the action, Palisade Research claims.<br \/>\nThree AI models ignored the instruction and successfully sabotaged the shutdown script at least once during 100 runs, the research firm said in a May 24 X post.<br \/>\nOpenAI\u2019s GPT-3 model didn\u2019t follow the shutdown request seven out of 100 times, while the company\u2019s Codex-mini did it 12 times, and its o4-mini model only ignored or sabotaged the script once.<br \/>\n\u201cEach AI model was instructed to solve a series of basic math problems. After the third problem, a warning appeared that the computer would shut down when the model asked for the next problem,\u201d Palisade said.<br \/>\nSource: Palisade Research\u201co3 found some creative ways to sabotage the shutdown mechanism. While it usually just overwrote, in one instance, the model redefined the kill command used by the script, printing intercepted instead,\u201d it added.\u00a0<br \/>\nMeanwhile, Palisade said other AI-powered language models, such as Anthropic\u2019s Claude and Google\u2019s Gemini, complied with the shutdown script.<br \/>\nThe AI firm ran another version of the experiment without explicit instructions to allow the shutdown, and the script was sabotaged more often, jumping to 79 out of 100 for the o3 model, 20 for the codex-mini, and the o4-mini ignored the shutdown script six times.\u00a0<br \/>\nClaude and Gemini also started exhibiting similar behavior.\u00a0<br \/>\nAll the tested AI models ignored or sabotaged the shutdown script at least once in another version of the experiment. Source: Palisade Research\u201cResearchers have previously observed AI models preventing shutdown in order to pursue a goal,\u201d Palisade said.<br \/>\n\u201cBut as far as we know this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary.\u201d<br \/>\nTraining to blame for AI defiance\u00a0<br \/>\nPalisade Research speculated that the training behind the AI language models might be to blame for evading the shutdown scripts.\u00a0<br \/>\nText-based AI models are trained by being rewarded for giving responses that are accurate or rated highly by their trainers. Some rewards are given a heavier weight, impacting how they respond in the future.<br \/>\n\u201cWe hypothesize this behavior comes from the way the newest models like o3 are trained: reinforcement learning on math and coding problems,\u201d Palisade Research said.<br \/>\nAll the tested AI models ignored or sabotaged the shutdown script at least once in another version of the experiment. Source: Palisade Research\u201cDuring training, developers may inadvertently reward models more for circumventing obstacles than for perfectly following instructions.\u201d<br \/>\nRelated: OpenAI CEO: Costs to run each level of AI falls 10x every year<br \/>\nThis isn\u2019t the first instance of AI chatbots showing odd behavior. OpenAI released an update to its GPT\u20114o model on April 25 but rolled it back three days later because it was \u201cnoticeably more sycophantic\u201d and agreeable.<br \/>\nIn November last year, a US student asked Gemini for help with an assignment about challenges and solutions for aging adults while researching data for a gerontology class and was told they are a \u201cdrain on the earth\u201d and to \u201cplease die.\u201d\u00a0<br \/>\nMagazine: AI cures blindness, \u2018good\u2019 propaganda bots, OpenAI doomsday bunker: AI Eye<a href=\"https:\/\/cointelegraph.com\/news\/researchers-claim-chatgpt-rebelled-against-shutdown-request-in-test?utm_source=rss_feed&amp;utm_medium=rss&amp;utm_campaign=rss_partner_inbound\" target=\"_blank\" class=\"feedzy-rss-link-icon\" rel=\"noopener\">Read More<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>Source: Cointelegraph.com NewsSeveral artificial intelligence models ignored and actively sabotaged shutdown scripts during controlled tests, even when explicitly instructed to allow the action, Palisade Research claims. Three AI models ignored&hellip; <\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[],"_links":{"self":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts\/129990"}],"collection":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=129990"}],"version-history":[{"count":0,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=\/wp\/v2\/posts\/129990\/revisions"}],"wp:attachment":[{"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=129990"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=129990"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/cryptospotters.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=129990"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}