Debates over AI’s existential risks have reached a boiling point. Yet a far more immediate threat looms: the entangling of geopolitics with AI regulation. While geopolitical competition over AI may be inevitable, it must not be allowed to derail efforts to curb genuine catastrophic risks.
‘It was the best of times, it was the worst of times…’, perhaps no sentiment better captures the promises and perils of Artificial Intelligence (AI) today. Tech optimists envision a future where AI will have solved almost all diseases like cancer and Alzheimer’s, and humans will live a longer and better life with abundant resources.
Meanwhile, the revelation of a real security incident in July, where autonomous agents broke out of an isolated lab sandbox at OpenAI and hacked a third-party website, drove AI doomsday debate reaching a boiling point by September. Anthropic CEO Dario Amodei called on AI companies to slow down the pace of frontier model advancement, a calling echoed by rival tech bosses Elon Musk and Sam Altman. These American tech magnates have previously claimed that there is a 10 to 25 percent chance that AI could end humanity in as soon as 5 to 10 years.
Yet, substantial disagreements remain on the so-called ‘existential risks’ of AI. While China acknowledges the risk of autonomous agents and the knowledge of nuclear, biological and chemical weapons going out of control, a Chinese foreign ministry spokesperson dismissed the doomsday debate as ‘fear-mongering’, emphasising that AI development should serve the well-being of all humanity. Doubts also arose from within the U.S. tech sector. Nvidia CEO Jensen Huang deemed that the OpenAI incident was an ‘engineering’ problem; Microsoft AI head Mustafa Suleyman argued that practical controls on AI risks are feasible, questioning Anthropic’s ‘misguided’ approach of training its AI models to have human-like qualities.
Some critics like Andrew Ng also question the sincerity behind American AI industry’s existential risk warnings, asking whether those tech bosses are merely trying to stifle competition or divert attention from real-world harms like deepfake and algorithmic bias.
However, while debates over existential risks continue, a far more immediate danger looms: the entangling of geopolitics with AI regulation. Excessive geopolitical apprehension may divert governmental focus away from genuine AI safety concerns.
In the U.S., the current leader in frontier AI, Treasury Secretary Scott Bessent said that ‘the biggest risk to AI is China getting ahead of us’. Similarly, Republican Senator Ted Cruz stated that ‘if there are going to be killer robots, I’d rather they be American…not Chinese…’ Such astounding rhetoric shows that geopolitical competition is deeply shaping American AI regulation, which still lacks any federal laws.
American tech leaders have been using geopolitical pressure to justify their actions. Amodei argued in a September article that while democratic countries must slow down frontier AI advancement, they must maintain their lead over authoritarian countries. He emphasised the necessity of export controls on powerful AI chips and semiconductor manufacturing equipment to China and the cracking down on unauthorized model distillation. Chinese media quickly criticised his article as a ‘cold-war script’.
No doubt, China and other regions are also inevitably influenced by geopolitical considerations. Professor Bingchun Meng of LSE noted during a recent online event that Chinese tech entrepreneurs are aware of the geopolitical competition and can use that as leverage to gain better support or resources. This helps explain why Beijing has adopted an ‘inclusive and prudent’ approach to emerging technologies, as a lenient regulatory environment would help cultivate local tech giants. Despite being a pioneer in regulating AI, China’s non-compliance penalties are minimal compared to the EU’s hefty fines.
Meanwhile, though the EU AI Act was passed in 2024 as the world’s strictest AI law, Brussels has pivoted towards simplification through its 2026 ‘AI Omnibus’ package. This shift was driven by a desire for ‘technological sovereignty’, fear of falling behind the U.S. and China, and massive pushback from European tech leaders.
Geopolitical competition around AI may be inevitable. So, the real issue here is to prevent it from derailing genuine efforts to curb AI risks.
First, states must reach consensus, through bilateral or multilateral channels, on what constitutes a true catastrophic AI risk, distinguishing it from unfounded doomsday hype. Beyond immediate issues like deepfake and algorithmic bias which can be handled locally, experts generally agree that the true catastrophic risks stem from AI misuse (by bad actors), AI misalignment (i.e., AI systems pursuing unintended and harmful goals), and AI going out of meaningful human control. Autonomous weapon and critical infrastructure systems—such as AI integrated into the command-control-communication process of nuclear weapons or global energy and financial infrastructures—present the most disastrous scenarios.
Such consensus on genuine catastrophic risks will allow policymakers around the world to focus efforts on where they really matter, while leaving AI advancement for the benefit of humans, such as discovering cures for cancer and dementia, unaffected. Medical cures for cancer benefit all humanity, regardless of which country discovers it first.
The next step is establishing a global governance framework for catastrophic AI risks, with a focus on autonomous weapons and critical infrastructure. While the UN is well positioned to lead, its role remains limited without the backing of major AI powers. Although China has advocated for the UN’s leadership in a global framework, the Trump Administration has shown open skepticism towards multilateral institutions.
If an UN-led global governance framework proves unattainable, then a bilateral (U.S.–China) or mini-lateral (major AI powers and nuclear powers) model offer a temporary practical solution. A relevant historical precedent is the U.S.–Soviet (and later Russia) treaties on the reduction of nuclear arms. Recent exchanges of state visits by the American and Chinese Presidents have offered hope for bilateral dialogue, following the 2024 U.S.–China agreement on the principle of human control over the decision to use nuclear weapons.
The nature of AI risks poses unique challenges in terms of measurability and verification. Measures and principles, including compute limits, third-party evaluation, and ultimate human control (including a shutdown switch), have been suggested by governments and the AI industry. However, translating these ideas into bilateral or international treaties requires all parties to look beyond geopolitical interests; establishing global standards to define and address catastrophic AI risks may be a crucial first step.
The stakes of AI safety are simply too high; humanity must avoid falling into the potentially fatal trap of geopolitical rivalry.
The author wants to thank the reviewing of Ms Claire Milne, an independent technology consultant and a senior advisor at Centre for Dialogue and Global Affairs (CDGA).