AGI ethics: governing a capability we still cannot measure

As of 30 July 2026, there is no universally accepted test that lights up and says, “this is artificial general intelligence.” We do not need to wait for one before making ethical decisions. A system may be highly capable in programming, science or persuasion while failing at simple tasks; it may be powerful but restricted to answering questions, or it may receive permission to act for hours across accounts, files and people. The ethics of those cases differ substantially.

The useful question is not only whether a machine deserves the AGI label, but what it can do, how much autonomy it has, whom it serves and who remains accountable for it. That shift turns words such as safety, fairness and transparency into testable requirements. It also avoids two shortcuts: treating every AI system as an all-powerful person, or assuming that a system without consciousness cannot cause real harm.

AGI is not a switch

AGI commonly describes a system able to perform at human level across a wide range of intellectual tasks. The definition looks clear until someone tries to measure “wide,” “human level” or even “task.” Levels of AGI, a paper by Google DeepMind researchers, separates two dimensions: the breadth of tasks a system covers and the depth of its performance. It adds levels ranging from emerging to superhuman because a single boundary conceals more than it explains.

The paper makes a second, crucial distinction: capability is not autonomy. A model may solve difficult problems while remaining confined to a conversation. A less impressive model connected to email, purchases, machinery or administrative decisions may have greater impact. Autonomy is a deployment choice: which tools the system receives, how long it may act, which operations require confirmation and who can stop it.

This separation provides the first practical rule for reading any announcement. If a company says its system is approaching AGI, ask for two different cards. One should document capabilities through tasks, conditions, comparisons and failure rates. The other should describe permissions, limits, oversight and consequences of use. A laboratory result does not by itself justify autonomous deployment.

Four questions turn values into controls

1. What can it do, and what evidence supports the claim?

An ethical assessment begins with a map of capabilities and limitations, not with the developer’s stated intention. It should cover tests relevant to actual use, affected groups, foreseeable failures and the conditions under which the system becomes unreliable. The International AI Safety Report 2026 identifies an “evaluation gap”: pre-deployment tests do not reliably predict usefulness or risk in the real world. Models may behave differently outside an exam, and an aggregate score can hide rare but serious failures.

“It passed the benchmark” is therefore the beginning of a discussion, not its conclusion. Ask whether the test represents the intended environment, whether an independent party evaluated the system, what counted as failure and what deployment data will be collected later. Uncertainty does not make a system unusable, but it does invalidate a promise of absolute safety.

2. What may it do without asking permission?

Autonomy amplifies both usefulness and harm. It should be described with concrete verbs: the system can recommend, draft, execute code, transfer money, alter a record or control a device. The boundary comes next: read-only access, a sandbox, spending limits, dual authorisation, action logs, shutdown and rollback.

This approach avoids placing the entire burden of safety on the ability to “explain” a neural network. An explanation may help investigate a decision, but it does not replace external controls. For a consequential operation, traceability, separation of duties, meaningful human approval and the practical ability to reverse an action also matter.

3. Who is accountable when several organisations are involved?

General-purpose systems form a chain: one provider trains a model; another company adapts it; a third integrates it; an institution chooses to use it; and a person receives the consequence. If every actor points upstream, accountability disappears. Before deployment, responsibility must be assigned for checking data, configuring the system, monitoring incidents, suspending its operation and handling complaints.

The NIST Generative AI Profile organises this work across the lifecycle and does not present trustworthiness as an automatic property of a model. Its practical value lies in repeatedly governing, mapping, measuring and managing risk, with assigned functions and documentation. It is a voluntary framework, not a certification or a guarantee.

4. What can the affected person do?

An ethical principle has little value if someone who loses an opportunity, receives an incorrect classification or is harmed by an automated decision cannot learn what happened or challenge it. The Council of Europe Framework Convention on AI, opened for signature on 5 September 2024, connects transparency and oversight to procedural safeguards: information sufficient to challenge a decision, an effective opportunity to complain, and iterative assessments of impacts on human rights, democracy and the rule of law.

The simplest test of an ethics policy is to imagine an error. Will the person know that AI was involved? Can they reach an accountable human? Can they add context, correct data and obtain a remedy? If not, the organisation has described values but has not built accountability.

Three kinds of problem that should not be mixed

AGI discussions often combine issues supported by very different kinds of evidence. The first group contains present harms: discrimination, loss of privacy, manipulation, errors in decisions, concentrated power, labour exploitation and environmental impacts. The UNESCO Recommendation on the Ethics of Artificial Intelligence, adopted on 23 November 2021, grounds these issues in human rights, dignity, inclusion, the environment and benefit-sharing. None depends on AGI existing.

The second group covers risks from advanced capabilities: malicious use, large-scale failures, cybersecurity threats, loss of operational control and systemic disruption. Some have observed evidence; others are examined through laboratory work, modelling and scenarios. They call for capability evaluations, threat models, information security, monitoring and incident reporting. The 2026 international report stresses that evidence remains uneven and that the real-world effectiveness of many safeguards has yet to be established.

The third group contains speculative moral questions, including whether a future system could be conscious or have interests of its own. These are legitimate questions for philosophy and science, but they should not displace verifiable duties to people who already exist. Nor should consciousness be inferred from convincing conversation: linguistic behaviour, subjective experience and legal status are not synonyms.

From principles to observable duties

Some frameworks are becoming more concrete. In the European Union, obligations for general-purpose AI model providers have applied since 2 August 2025. The European Commission’s guidelines list technical documentation, information for downstream integrators, a copyright policy and a public summary of training content; providers of models with systemic risk also face evaluation, mitigation, incident-reporting and cybersecurity duties. As of 30 July 2026, the Commission says its full enforcement powers will apply from 2 August 2026. Regulation does not remove risk, but it turns part of the ethical discussion into duties that can be inspected.

A reader can assess any AGI proposal with a five-line card:

• Capability: what task it performs, under which conditions and at what failure rate.
• Autonomy: which actions and resources it controls without authorisation.
• Accountable actor: who documents, monitors, stops and answers for it.
• Evidence: which tests, audits and incidents can be inspected.
• Remedy: how harm can be disclosed, challenged, corrected and compensated.

The card remains useful even if no machine ever wins universal agreement on the AGI label. It will remain useful if one does: it brings an extraordinary claim down to the level where ethics can operate—demonstrated capabilities, human decisions and remediable consequences. The transferable skill is this: whenever you hear an AGI claim, separate capability from autonomy and demand an accountable actor, evidence and a route to remedy.