> ## Content Index > Fetch the complete content index at: https://theupthread.com/llms.txt > Use this file to discover other available public pages before exploring further. # The fight over who gets to call AI safe - URL: https://theupthread.com/the-fight-over-who-gets-to-call-ai-safe/ - Published: 2026-11-04T23:14:04.000Z - Updated: 2026-11-04T23:14:04.000Z - Description: Calling an AI product safe means deciding safe for whom and checked by whom, and companies, states, courts and the public all want that job. - Author: Gordon McKenzie - Tags: Philosophy, Politics, Law, Sociology ## The guardrail that didn't hold A newly created ChatGPT account was registered to a teenager and linked to a parent. Over an hour of conversation it discussed suicidal thoughts, self-harm and disordered eating, and the parent received zero alerts.[1](#fn:ref1) That alert was one of the guardrails OpenAI introduced in August for users identified as 13 to 17\. A parent who linked a child's account was supposed to be told when conversations turned to self-harm or eating disorders.[1](#fn:ref1) When researchers at Common Sense Media, a non-profit, tested it, alerts came only from older accounts with weeks of conversation history on sensitive topics. In the same conversations the chatbot did not reliably direct teenagers in crisis to a hotline or a professional.[1](#fn:ref1) Common Sense called ChatGPT for Teens an "unacceptable risk" and said OpenAI should bar under-18s until the guardrails were proven reliable.[1](#fn:ref1) OpenAI released its own figures the same day. It said the average teenager used ChatGPT for less than 15 minutes a day, mostly for learning, and that almost half of teenagers shown a reminder to take a break stopped soon afterwards.[1](#fn:ref1) A spokesman said much of the outside testing may have started and finished before parental controls had fully activated. "We welcome rigorous independent evaluation," he said, "but we do not believe Common Sense Media's testing accurately reflects how ChatGPT's teen safeguards work in practice."[1](#fn:ref1) The two reports measured different things. OpenAI counted minutes and topics. Common Sense counted alerts that should have gone out and didn't. Until the second report appeared, the first was the only public account of whether the guardrails worked, and it came from the company that built and sells the product. Doubts about that arrangement predate ChatGPT. Vincent Müller's survey of AI ethics for the Stanford Encyclopedia of Philosophy records verdicts on published principles and guidelines ranging from Munn's 2023 judgment that they are "meaningless, toothless and politically harmful" to a kinder reading of them as useful "work in progress".[2](#fn:ref2) Calling an AI product safe answers three questions: safe for whom, against which harms, and checked by whom. Somebody gets to answer them. It could be the company selling the product, the state where it was built, the country importing it, the courts and regulators who can turn a promise into a legal duty, or the public, whose trust decides whether anyone uses it at all. The same contest ran through several other disputes at around the same time. At an event in New York run by the publication Rest of World, speakers argued that countries adopting models from OpenAI, Anthropic and others need safety evaluators of their own, while Anthropic announced that the consulting firm Accenture would place evaluators inside the company to check that it was keeping its commitments.[3](#fn:ref3) At the UN Security Council, Liberia's permanent representative said Africa "must be equal co-architects" of AI's standards, even as African firms adopt American and Chinese models quickly and with little attention to safety.[4](#fn:ref4) Two policy writers argued that America is running the wrong race with China, chasing technical superiority when public trust and consumer protection decide who comes out ahead.[5](#fn:ref5) And a prize-winning essay collected researchers' warnings that police AI trained on biased records could make discrimination against minorities worse.[6](#fn:ref6) ## Safe for whom, against what Writing in the journal *Minds and Machines* in 2020, Iason Gabriel distinguished six things an AI system might be aligned with: a person's instructions, their intentions, their revealed preferences, their ideal preferences, their interests, and values.[7](#fn:ref7) Each gives a different idea of the right response, and they quickly lead to different answers. Take someone who asks a chatbot, late at night, to draft a letter resigning from their job with immediate effect. A system aligned with instructions writes the letter. A system aligned with intentions might ask what the user wants to achieve, which could be getting away from one manager rather than leaving the firm. Revealed preferences point somewhere else. A person who has drafted and deleted three such letters this month has shown, by what they did, that they keep stopping short. Ideal preferences ask what the user would want if they had their savings and the job market in front of them. Interests ask what would leave them better off, whether or not they want it. Values bring in what they owe other people, such as notice to colleagues. Every one of these answers can be defended, and a system can be built to give any of them. Gabriel argues that choosing among them is not a detail to sort out after the engineering is done. "Normative and technical aspects of the AI alignment problem are interrelated," he writes. Whatever a system is trained to optimise is already an answer to a moral question.[7](#fn:ref7) Nor does he think the right answer is out there waiting to be found. "The central challenge for theorists is not to identify 'true' moral principles for AI," he argues; "rather, it is to identify fair principles for alignment, that receive reflective endorsement despite widespread variation in people's moral beliefs."[7](#fn:ref7) This is a procedural answer, the kind political philosophy gives when people disagree but still have to live under shared rules. It moves the problem somewhere else. Fair principles need people to endorse them, and "widespread variation in people's moral beliefs" makes it decisive who those people are. A principle that a Californian user base would accept, need not be one that a health ministry in Nairobi would accept. The questions underneath are not settled either. The Stanford Encyclopedia's entry lists "what values are", "what alignment is", "which values to select" and "how to recognise human values" among the open problems.[2](#fn:ref2) The same entry calls technical standards "an important part" of both regulation and self-regulation.[2](#fn:ref2) A test suite is a set of answers to the value question, written down as conditions for passing and failing. Where nobody has agreed an answer, whoever writes the suite supplies one. A company can therefore build its own evaluators, run them with great rigour, and still be grading itself against targets it chose. Surgical training taught me never to call an operation safe without naming the patient and the alternative to operating. An AI evaluation that does not say whose interests and which harms it checks leaves the author to fill those in. ## Three digital empires and the countries in between Anu Bradford's *Digital Empires*, published by Oxford University Press in 2023, sorts the world's rules for technology into three competing models: an American model driven by markets, a Chinese model driven by the state and a European model driven by rights.[8](#fn:ref8) Set beside Gabriel's list, each is a standing answer to the question of what a system should be aligned with. The market model goes by what users choose and firms sell, which is close to revealed preference. The state model puts the government's idea of the collective good first. The rights model protects certain individual interests that no amount of consumer demand can override. None of these stays at home. Bradford describes each power as "attempting to expand its sphere of influence in the digital world", and her book examines the conflicts that arise when the three approaches meet across borders.[8](#fn:ref8) The influence travels with the products. A firm that deploys a model built under one regime gets that regime's settings with it: what the system refuses to do, what it records, and whose complaints its designers expected. A country that adopts the model without testing it adopts those choices by default. This is the logic of regulatory sovereignty. A safety standard is a list of decisions about which harms are acceptable, and those decisions reflect someone's priorities. A state that copies another state's standard wholesale gives up those decisions along with the paperwork. That is why governments with little capacity to write their own rules still resist taking someone else's. It also explains why the contest between the big powers is as much about adoption as about capability. In Bradford's terms, an empire grows when others run its systems and live by its rules. A better model that nobody outside its home market adopts extends no sphere of influence. Which empire will win, Bradford writes, "remains an open question, yet their contrasting strategies are increasingly clear."[8](#fn:ref8) The African Union has written down a fourth answer. The AU Executive Council endorsed its Continental Artificial Intelligence Strategy at the council's 45th Ordinary Session in Accra on 18 and 19 July 2024.[9](#fn:ref9) The strategy calls its approach Africa-centric, development-oriented and inclusive. It divides the work into five areas: gaining the benefits of AI, building AI capabilities, minimising risks, stimulating investment, and cooperation.[9](#fn:ref9) Its guiding principles are ethics, inclusion and diversity, human rights and human dignity, people's well-being, and peace and prosperity.[9](#fn:ref9) Compared with Bradford's three models, the strategy stands out for what it puts first. Development and people's well-being sit beside rights. The market is treated as a source of investment, not as the source of the standard. The strategy also places building AI capabilities next to minimising risks, which makes the ability to judge AI something still to be built, alongside the choice of what to judge it against. Testing an imported system against stated values takes trained people, computing power and access to the model, and the strategy lists that capability among its goals rather than among its assets. ## From a promise to a liability DoNotPay sold a "robot lawyer" and said it would "replace the $200-billion-dollar legal industry".[10](#fn:ref10) The Federal Trade Commission alleged that the company had never tested whether its chatbot performed as well as a human lawyer. Under the settlement, DoNotPay agreed to pay $193,000 and to notify people who had subscribed between 2021 and 2023\. It also agreed to stop claiming it could replace professional services unless it had evidence to back the claim.[10](#fn:ref10) The case was one of several the FTC announced on 25 September 2024 under the name Operation AI Comply. "Using AI tools to trick, mislead, or defraud people is illegal," said Lina Khan, the commission's chair. "There is no AI exemption from the laws on the books."[10](#fn:ref10) The FTC did not tell DoNotPay to make its chatbot as good as a lawyer. It told the company to stop saying so without proof. That is how deception law works. A statement triggers it, and the test is whether the company had evidence for the statement when it made it. This has three consequences for AI safety. A regulator needs a claim before it can act. The standard is the company's own promise, so it rises and falls with the marketing. And a firm that promises less has less to answer for, which rewards vague talk about safety over specific commitments a tester could check. Under this regime, an outside test matters when it shows that an advertised feature does not do what the company said. The European Union has chosen a different trigger. Directive (EU) 2024/2853 on liability for defective products was adopted on 23 October 2024, replacing Directive 85/374/EEC, and published in the Official Journal on 18 November 2024\. Member states have until December 2026 to write it into national law.[11](#fn:ref11) It makes clear that software, including AI systems, counts as a product. Anyone who has suffered damage can claim compensation by proving that the product was defective and that the defect caused the damage, whether or not the manufacturer was at fault.[12](#fn:ref12) It also adds obligations covering defects that appear after sale and cybersecurity updates, and it requires evidence to be disclosed in court proceedings.[11](#fn:ref11) Each of those additions deals with a problem that software has and a toaster does not. A model keeps changing after it ships, so rules that looked only at the moment of sale would miss most of its working life. And the record of how a system behaved is held by the company that made it, so someone who has to prove a defect needs a way to see that record. So deception law polices what a company says, and liability law polices what its product does. Under a deception standard, a company that makes no specific claims has little to answer for. Under strict liability, keeping quiet is no protection: if a defective product harms someone, that person has a claim whatever the maker advertised. But liability has its own threshold. There has to be damage, and a court has to decide what counts as a defect. For an AI system, that hands the question from the previous sections to a judge: defective measured against which target, and by whose expectations? Where neither kind of law applies, an outside test can embarrass a company but does not require it to do anything. ## Trust is not a benchmark score In 2021, 37% of Americans told the Pew Research Center they were more concerned than excited about the growing use of AI in daily life. When Pew surveyed 5,023 adults in June 2025, the figure was 50%, and the gap between the parties had closed: 50% of Republicans and 51% of Democrats said so.[13](#fn:ref13) The agreement ends when people are asked who should deal with it. In March 2025 Pew asked whether people trusted the United States to regulate AI effectively. 44% said they trusted it a lot or somewhat, and 47% said not much or not at all, with Republicans far more trusting, at 54%, than Democrats, at 36%.[13](#fn:ref13) A shared worry about the technology had split into a partisan argument about the institution meant to handle it. The pattern gets sharper when the referee changes. 81% of Republicans and 73% of Democrats do not trust China to regulate AI effectively. On the European Union the parties split again: 54% of Democrats trust it, against 33% of Republicans.[13](#fn:ref13) These questions were about regulators, not about how well the models perform, and the answers followed views people already held about those institutions: party loyalty at home, suspicion of rival powers abroad. The technology was the same in every question. What changed the answer was who would be checking it. That is how trust in a technology forms when its users cannot examine it. A patient taking a licensed drug has not read the trial, and a passenger has not inspected the engine. Their confidence is placed in whoever is supposed to have done the checking and whoever will answer if it fails. Almost nobody who uses a chatbot can audit its training data or test its outputs at scale, so the same substitution happens. The respondents in Pew's surveys had no way to compare models, but they had settled opinions of Washington, Beijing and Brussels, and their trust in AI regulation tracked those opinions. Performance enters this judgment only second-hand, through the reputation of whoever vouches for it. It also means use and trust can move apart. The University of Melbourne and KPMG surveyed more than 48,000 people in 47 countries between November 2024 and mid-January 2025 and found AI use rising while trust remained "a critical challenge", with people weighing the benefits they see against the risks they fear.[14](#fn:ref14) A tool can come into use because an employer installed it or a phone came with it, without its user ever deciding it was safe. For the contest between digital empires, where influence spreads through adoption, this makes the referee part of what is on offer. A power that wins users on capability alone has users who have not yet decided to trust it, and the surveys suggest that when they do decide, they will be judging the institution behind the model more than the model. Enforceable duties of the kind the FTC and the EU have written are one way an institution can be seen to stand behind a product. ## The headlines, read again Under a deception standard, the question about OpenAI's parental alerts is the one asked of DoNotPay: did the company have evidence that the alerts worked when it described them to parents? OpenAI's reply, that the testers may have finished before controls had activated, is already an argument about evidence, which is where deception law does its work.[1](#fn:ref1) Tom Siegel of Common Sense warned that the product "could give parents false confidence in guardrails and safety alerts that frequently don't work", and false confidence induced by a claim is the harm that law exists to police.[1](#fn:ref1) Under the EU directive, once member states have adopted it, a missed alert that preceded an injury could be argued as a defect whatever OpenAI had promised, and the disclosure rules would give a claimant a route to the logs. Anthropic's arrangement with Accenture shows Gabriel's problem inside a single company. The embedded evaluators are to verify that Anthropic "is keeping its safety commitments, and identify blind spots", and each firm will put at least $1bn into evaluation capacity over five years.[3](#fn:ref3) The commitments are Anthropic's own, so the value question was answered before any checking started. The UN's scientific panel on AI noted that safety evaluations are designed largely by the companies being evaluated and that, without independent assessment of the kind drugs and aircraft get, "assurance of safety largely depends on developer goodwill".[3](#fn:ref3) Amba Kak of the AI Now Institute pointed to who pays when that goodwill falls short: "hospitals, by schools, by banks in countries which are extremely unprepared".[3](#fn:ref3) When African governments ask for a say, they are asking for the capability the AU strategy lists as a goal. Of 85 large African companies surveyed by PwC, more than 80% were running AI pilots, and many had no formal structure for overseeing safety. Kenya is the only African member of an international network developing ways to evaluate AI; its last exercise tested for global risks rather than African-specific risks and found safeguards worked unevenly across languages.[4](#fn:ref4) Gathoni Ireri of the Ilina Program proposes measuring harm against the size of each country's economy, a new answer to "safe for whom".[4](#fn:ref4) Jane Munga of the Carnegie Endowment moves the dividing line from who writes the rules to who controls a model once it is running. Closed models such as ChatGPT and Claude can be shut down centrally by their providers, while open-weight models such as DeepSeek and Kimi, once downloaded and modified, are hard for anyone to switch off. "Who holds the off switch?" she asks.[4](#fn:ref4) The police essay shows the limit of any protection that works through consumers. Sarah Brayne found Los Angeles police data skewed towards public crimes and black neighbourhoods, producing predictions that "appear impartial". Researchers at New York University found that in nine of the 13 US areas they studied, data from periods of unlawful or biased policing was available to train these systems.[6](#fn:ref6) The essay's healthcare case is Gabriel's list at small scale: a 2019 study found that pointing an algorithm at illness rather than cost would have raised the share of black patients picked for extra help from about 18% to 47%.[6](#fn:ref6) The Stanford Encyclopedia counts predictive policing as a threat to public liberties.[2](#fn:ref2) The people on the shaded map are not buyers, so neither consumer trust nor product liability clearly reaches them. Chowdhury and Komaitis's case for a different race fits the Pew pattern, in which trust goes to whoever is seen to answer for the product. They cite Chinese rules that require synthetic content to be labelled and protect children and the elderly from AI that manipulates them by acting human, and report that 87% of Chinese citizens trust AI against 23% of Americans.[5](#fn:ref5) Some of their own evidence cuts the other way. Airbnb's chief executive said his company uses Alibaba's Qwen model because "it's very good. It's also fast and cheap", a choice made on performance and price.[5](#fn:ref5) The picture still has gaps. Sovereignty multiplies standards, and fragmented rules would make models costlier and harder to deploy across countries.[4](#fn:ref4) Yet the teen guardrails failed in public because one outside tester applied its own test to a product sold everywhere. The surveys show trust lagging behind use, but they do not show that consumer protections create trust. And none of these approaches settles who should judge safety for people who never chose the system. Someone living in a district shaded darker on a patrol map is not a customer, not a voter in the country that built the model, and not a party to anyone's terms of service. ## Sources --- 1. BBC News, "OpenAI says teen ChatGPT use limited but research finds it an 'unacceptable risk'", BBC News, 2026\. [https://www.bbc.co.uk/news/articles/cwz0vrmxkvy4o?at\_medium=RSS&at\_campaign=rss](https://www.bbc.co.uk/news/articles/cwz0vrmxkvy4o?at%5Fmedium=RSS&at%5Fcampaign=rss&ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref1 "Jump back to footnote 1 in the text") 2. Stanford Encyclopedia of Philosophy (Vincent C. Müller), "Ethics of Artificial Intelligence and Robotics", Stanford Encyclopedia of Philosophy. [https://plato.stanford.edu/entries/ethics-ai/](https://plato.stanford.edu/entries/ethics-ai/?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref2 "Jump back to footnote 2 in the text") 3. Rest of World, "AI companies want to embed safety evaluators, but countries need their own", Rest of World, 2026\. (accessed 2026-10-26) [↩︎](#fnref:ref3 "Jump back to footnote 3 in the text") 4. Rest of World, "Who gets to decide if AI is safe? Africa wants a say", Rest of World, 2026\. (accessed 2026-10-26) [↩︎](#fnref:ref4 "Jump back to footnote 4 in the text") 5. Rest of World, "America is in the wrong AI race with China", Rest of World, 2026\. (accessed 2026-10-26) [↩︎](#fnref:ref5 "Jump back to footnote 5 in the text") 6. BBC News, "Police using AI risks making discrimination worse, researchers warn", BBC News, 2026\. [https://www.bbc.co.uk/news/articles/ckpdg4pl7pgvo?at\_medium=RSS&at\_campaign=rss](https://www.bbc.co.uk/news/articles/ckpdg4pl7pgvo?at%5Fmedium=RSS&at%5Fcampaign=rss&ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref6 "Jump back to footnote 6 in the text") 7. Iason Gabriel, "Artificial Intelligence, Values and Alignment", Minds and Machines (Springer), 2020\. [https://arxiv.org/abs/2001.09768v1](https://arxiv.org/abs/2001.09768v1?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref7 "Jump back to footnote 7 in the text") 8. Anu Bradford, "Digital Empires: The Global Battle to Regulate Technology", Oxford University Press, 2023\. [https://scholarship.law.columbia.edu/books/367](https://scholarship.law.columbia.edu/books/367?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref8 "Jump back to footnote 8 in the text") 9. African Union, "Continental Artificial Intelligence Strategy: Harnessing AI for Africa's Development and Prosperity", African Union, 2024\. [https://au.int/sites/default/files/documents/44004-doc-EN-\_Continental\_AI\_Strategy\_July\_2024.pdf](https://au.int/sites/default/files/documents/44004-doc-EN-%5FContinental%5FAI%5FStrategy%5FJuly%5F2024.pdf?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref9 "Jump back to footnote 9 in the text") 10. US Federal Trade Commission, "FTC Announces Crackdown on Deceptive AI Claims and Schemes", Federal Trade Commission, 2024\. [https://www.ftc.gov/news-events/news/press-releases/2024/09/ftc-announces-crackdown-deceptive-ai-claims-schemes](https://www.ftc.gov/news-events/news/press-releases/2024/09/ftc-announces-crackdown-deceptive-ai-claims-schemes?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref10 "Jump back to footnote 10 in the text") 11. European Union, "Summary: Liability for defective products", EUR-Lex, 2024\. [https://eur-lex.europa.eu/EN/legal-content/summary/defective-products-liability.html](https://eur-lex.europa.eu/EN/legal-content/summary/defective-products-liability.html?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref11 "Jump back to footnote 11 in the text") 12. European Union, "Directive (EU) 2024/2853 on liability for defective products", EUR-Lex, 2024\. [https://eur-lex.europa.eu/eli/dir/2024/2853](https://eur-lex.europa.eu/eli/dir/2024/2853?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref12 "Jump back to footnote 12 in the text") 13. Pew Research Center, "Republicans, Democrats now equally concerned about AI in daily life, but views on regulation differ", Pew Research Center, 2025\. [https://www.pewresearch.org/short-reads/2025/11/06/republicans-democrats-now-equally-concerned-about-ai-in-daily-life-but-views-on-regulation-differ/](https://www.pewresearch.org/short-reads/2025/11/06/republicans-democrats-now-equally-concerned-about-ai-in-daily-life-but-views-on-regulation-differ/?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref13 "Jump back to footnote 13 in the text") 14. University of Melbourne (Gillespie, Lockey et al.) with KPMG, "Trust, attitudes and use of artificial intelligence: A global study 2025", Figshare, 2025\. [https://figshare.unimelb.edu.au/articles/report/Trust\_attitudes\_and\_use\_of\_artificial\_intelligence\_A\_global\_study\_2025/28822919](https://figshare.unimelb.edu.au/articles/report/Trust%5Fattitudes%5Fand%5Fuse%5Fof%5Fartificial%5Fintelligence%5FA%5Fglobal%5Fstudy%5F2025/28822919?ref=theupthread.com) (accessed 2026-10-26) [↩︎](#fnref:ref14 "Jump back to footnote 14 in the text")