By Vijeth Shivappa
Why governments must move beyond voluntary assurances and establish enforceable, technically credible oversight for increasingly autonomous artificial intelligence. The companies building increasingly autonomous AI systems should not be allowed to define, enforce and verify.
Artificial intelligence has entered an era of extraordinary contradiction. The companies developing the world’s most capable systems warn that AI could disrupt labour markets, amplify disinformation, enable cyberattacks and, in extreme cases, escape meaningful human control. Yet when governments propose binding safeguards, independent scrutiny or legal accountability, parts of the same industry caution that regulation could slow innovation. Both positions cannot be reconciled through voluntary promises alone. If AI is powerful enough to create systemic risk, it is powerful enough to warrant public oversight. And if its risks are not serious enough to justify oversight, repeated warnings of existential danger begin to look less like responsible disclosure and more like strategic messaging.
The central question is no longer whether AI should be regulated. It is whether the institutions building increasingly autonomous systems should also be permitted to define the limits of their own accountability.
Support in principle, resistance in practice
Few technology leaders openly argue for a lawless AI market. The preferred vocabulary is responsible, balanced or innovation friendly regulation. The disagreement begins when those principles become enforceable obligations.
Would developers accept mandatory independent evaluations before deploying high risk systems? Would they report serious safety incidents within a defined period? Would they preserve tamper evident records of consequential actions? Would they accept liability when foreseeable failures cause harm? Would they recognise the power of an external authority to suspend a system that cannot demonstrate adequate control?
Ethical principles without enforcement remain aspirations. Safety commitments without independent verification remain assurances. Governance becomes meaningful only when an institution outside the developer can establish that a boundary was crossed and require corrective action.
The economics of moving fast
AI development is a capital intensive global race. Companies are investing heavily in computing infrastructure, energy, data and specialist talent while competing for developers, enterprise customers and investor confidence. Safety reviews, controlled releases and independent audits add time and expense.
A company that delays deployment because of unresolved risks may lose ground to a competitor willing to release first. This is not necessarily evidence of bad intent. It is evidence of a misaligned incentive structure. The commercial rewards of speed accrue largely to the company, while the costs of failure can spread across workers, customers, public institutions and critical infrastructure.
Effective regulation can correct that imbalance by establishing a common safety floor. It can ensure that prudent companies are not commercially punished for exercising restraint.
Competition is not an exemption from responsibility
The strongest argument against stringent rules is geopolitical. American firms warn that excessive constraints could weaken the United States while competitors elsewhere accelerate. Similar arguments appear in other technology centres, each fearing that unilateral caution will transfer advantage abroad.
The concern is legitimate, but it cannot serve as a universal exemption from accountability. A country does not become strategically stronger by deploying systems it cannot reliably monitor, interrupt or contain. Unsafe AI can expose financial networks, energy systems, defence infrastructure and public services to manipulation or cascading failure.
Governance capacity is itself a strategic capability. The jurisdictions that learn to deploy advanced AI safely across critical sectors may achieve a more durable advantage than those that merely produce the largest models first.
Why self regulation is structurally insufficient
AI companies correctly observe that legislation moves slowly while technical capabilities change rapidly. Governments will need sustained access to specialist expertise, modern testing infrastructure and regulatory mechanisms that can adapt without rewriting primary legislation every year.
But regulatory difficulty is not an argument for regulatory absence. Aviation, pharmaceuticals, nuclear energy and global finance are technically complex, fast moving and consequential. None is governed solely by the voluntary assurances of its operators.
Industry expertise must inform regulation, but it cannot replace democratic authority. A voluntary commitment can be revised when leadership, funding or market conditions change. Safeguards that depend on corporate discretion are least dependable precisely when commercial pressure is greatest.
The accountability gap
Modern AI systems involve a chain of actors: foundation model developers, cloud providers, orchestration platforms, tool vendors, application builders and enterprise deployers. When an autonomous system causes harm, responsibility can dissolve across that chain.
The model provider may blame implementation. The deployer may cite unpredictable model behaviour. The application vendor may point to an external tool or data source. Each participant may claim control over only one layer, leaving the affected public with no clearly accountable party.
A credible framework must require answers before deployment: Who approved the system? What actions may it perform? Which data and tools may it access? When must it defer to a human? Can its actions be interrupted or reversed? What evidence will reconstruct a decision? And who is legally responsible when safeguards are missing or fail?
Without clear answers, responsible AI remains a communications position rather than an operating model.
Regulatory capture is also a danger
Not every corporate call for regulation necessarily advances the public interest. Large technology companies can absorb complex licensing requirements, legal processes and certification costs that smaller firms, universities and open source communities cannot. Rules intended to control dominant companies may inadvertently protect them from competition.
The remedy is not weaker oversight. It is proportionate oversight. Obligations should follow capability, autonomy, deployment context and potential harm. A low risk writing assistant should not face the same controls as an autonomous system influencing credit, medical treatment, industrial equipment or national infrastructure.
Good regulation must protect society without freezing the current market structure or allowing established companies to write rules that only they can afford to obey.
A global problem with fragmented institutions
AI does not respect regulatory borders. Models may be developed in one country, hosted in another and deployed through agents operating across several jurisdictions. A harmful action can move through global cloud, software and financial networks before any single authority understands the full chain.
The United States relies substantially on sector specific institutions. Europe has chosen a horizontal, risk based legislative approach. Other governments are developing national standards, voluntary codes or targeted rules. These approaches will differ, but they should converge around interoperable minimum safeguards for high risk systems.
Global alignment does not require identical laws. It requires shared expectations: independent evaluation, incident reporting, clear responsibility, secure agent identity, human override, runtime control and auditable evidence. Without interoperability, companies will face fragmented compliance while dangerous systems migrate toward the weakest jurisdiction.
Regulate consequential capability and action
Governments should not attempt to supervise every algorithm or software update. The strongest duties should apply where capability and deployment create credible pathways to serious harm.
A workable regime would combine risk classification, independent evaluation of frontier and high impact systems, mandatory reporting of serious incidents, continuous monitoring after deployment, verified emergency controls, strong identities for agents and tools, least privilege access, runtime authorization, tamper evident records and clearly allocated liability.
This approach avoids treating all AI as equally dangerous. It also recognises that risk is produced not only by what a model knows, but by what a connected system is empowered to do.
Governance must move to runtime
Much of today’s AI governance remains concentrated before release: training data reviews, red teaming, benchmark testing, model cards and policy documentation. These controls are necessary, but they are not sufficient for autonomous systems.
A model may behave acceptably in a laboratory and become dangerous when connected to live data, enterprise applications, payment networks, industrial controls or other agents. The most consequential risk may not reside inside the model. It may emerge from the permissions surrounding it.
The model is therefore no longer the sole unit of governance. Increasingly, the agent and the action it attempts must be governed. Every consequential request should be evaluated in real time against identity, authority, purpose, context, risk and policy. Unauthorized actions should be blocked. Ambiguous or high impact actions should be escalated to an accountable person. Each decision should produce durable evidence.
The principle is simple: the model must never have final authority. Human institutions must retain the power to define the boundary, enforce it in real time and verify that it held.
Trust is infrastructure for adoption
The debate is often framed as regulation versus innovation. That is the wrong choice. The real alternatives are governed innovation and uncontrolled deployment.
Weak oversight may accelerate short term experimentation, but one major failure involving healthcare, banking, energy, transport, defence or public administration could destroy confidence and provoke indiscriminate restrictions. Predictable rules can instead give developers clarity, enterprises confidence and citizens meaningful protection.
AI companies should contribute technical expertise and help develop workable standards. They should not possess a veto over democratic accountability. The institutions building powerful systems cannot simultaneously remain their sole rule makers, safety evaluators and judges.
The future of AI cannot rest only on the intentions of those racing to build it. It must rest on enforceable boundaries, independent oversight and verifiable proof that human authority remains intact.
