Framework Convention on Artificial Intelligence

Editorial note. This is a working draft published for comment. The registration threshold in Article 5 and the prohibited applications in Article 9 are the two provisions we expect to be argued about hardest, and the two we most want argued about.

The Parties to this Convention,

Recognizing that artificial intelligence systems are developed, hosted, and deployed across national boundaries, and that no Party acting alone can govern conduct that is distributed by design;

Concerned that the capabilities of advanced systems have repeatedly exceeded the expectations of those who built them, and that the interval between a capability appearing and its consequences being understood has been short;

Affirming that the protection of human rights, human dignity, and democratic governance does not weaken because a decision was produced by a machine;

Recalling the obligations of Parties under the International Covenant on Civil and Political Rights and under customary international law;

Desiring to establish a common floor of obligations that permits beneficial development while making the most serious risks answerable to public authority;

Have agreed as follows:

Part I — General Provisions

Article 1 — Objective

1. The objective of this Convention is to ensure that the development and deployment of advanced artificial intelligence systems is subject to effective public oversight, and that the risks arising from such systems are identified, disclosed, and mitigated before they materialize.

2. This Convention establishes minimum obligations. Nothing in it prevents a Party from adopting or maintaining stricter measures, provided those measures are otherwise consistent with its international obligations.

Article 2 — Definitions

For the purposes of this Convention:

a. “Artificial intelligence system” means a machine-based system that, for explicit or implicit objectives, infers from the input it receives how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.

b. “Advanced system” means an artificial intelligence system meeting the threshold established under Article 5, paragraph 2.

c. “Developer” means a natural or legal person that trains an advanced system, or that materially modifies such a system so as to change its capabilities.

d. “Deployer” means a natural or legal person that makes an artificial intelligence system available for use, or that uses it in respect of another person.

e. “Serious incident” means an event involving an artificial intelligence system that results in, or creates a substantial likelihood of, death or serious injury, substantial disruption to critical infrastructure, material interference with democratic processes, or serious infringement of fundamental rights.

f. “Independent evaluator” means a body accredited under Article 7 that has no financial interest in the system under evaluation.

Article 3 — Scope

1. This Convention applies to an advanced system where the Party is the state:

   a. in which the system is trained; b. in which the computational infrastructure used to train or operate the system is located; c. in which the developer or deployer is established; or d. in which the system is made available to persons or used in respect of persons.

2. Where more than one Party has jurisdiction under paragraph 1, each such Party retains its obligations under this Convention. Parties shall cooperate to avoid duplicative evaluation, but the existence of another Party’s jurisdiction is not a ground for declining to exercise one’s own.

3. This Convention does not apply to systems developed and used exclusively for purposes of national defense, save that Parties shall not invoke this paragraph in respect of systems deployed against their own populations.

Article 4 — General Principles

Parties shall give effect to this Convention in accordance with the following principles:

a. Burden of demonstration. The obligation to show that an advanced system is safe for its intended use rests with the developer and the deployer, not with the persons affected by it or with the public authority.

b. Effective oversight. Measures adopted under this Convention shall be capable of being verified by a party other than the one asserting compliance.

c. Non-discrimination. Obligations shall apply to comparable systems on comparable terms, irrespective of where the developer is established.

d. Proportionality. Obligations shall be calibrated to the severity and likelihood of the harm at issue.

Part II — Obligations of the Parties

Article 5 — National Registry of Advanced Systems

1. Each Party shall establish and maintain a registry of advanced systems trained, hosted, or deployed within its jurisdiction, and shall designate a competent national authority to maintain it.

2. A system falls within the registration requirement where:

   a. the cumulative computation used in training exceeds the threshold set by the Conference of the Parties under Article 11, paragraph 4; or b. the system is intended for deployment in a domain listed in Annex I (health care, criminal justice, employment, education, credit and insurance, and the operation of critical infrastructure); or c. the competent national authority determines, on reasoned grounds, that the system presents risks comparable to those in subparagraphs (a) or (b).

3. Registration shall be made before deployment and shall include the identity of the developer, a description of intended and foreseeable uses, the results of evaluation under Article 6, and the identity of any independent evaluator engaged.

4. Each Party shall make the registry publicly accessible, save for information whose disclosure would create a substantial security risk or reveal a trade secret. A Party withholding information under this paragraph shall publish the fact of the withholding and its legal basis.

Note on paragraph 2(a). A computation threshold is an imperfect proxy for capability and will age badly. It is used here because it is the only criterion presently available that a regulator can verify without the developer’s cooperation. Article 11(4) exists so that the threshold can be revised without reopening the treaty.

Article 6 — Pre-Deployment Evaluation

1. Each Party shall require that, before an advanced system is deployed, the developer conducts and documents an evaluation of:

   a. the system’s performance across the range of conditions in which it is intended to operate, including conditions at the edge of that range; b. its failure modes, and the consequences of each for affected persons; c. its susceptibility to misuse, including deliberate misuse by a user; and d. its differential performance across groups protected under the Party’s law.

2. The evaluation shall be conducted against criteria fixed before the evaluation begins. A developer shall retain records sufficient to permit the evaluation to be reproduced.

3. Where an evaluation identifies a risk that cannot be mitigated, the system shall not be deployed for the use giving rise to that risk.

Article 7 — Access for Independent Evaluators

1. Each Party shall establish a procedure for accrediting independent evaluators and shall require developers of registered systems to grant accredited evaluators such access as is necessary to verify the matters recorded under Article 6.

2. Access under paragraph 1 shall include access to the system in a form permitting meaningful evaluation, to evaluation records, and to documentation of training data provenance.

3. A Party shall protect an accredited evaluator from civil or criminal liability arising solely from good-faith evaluation conducted within the scope of its accreditation, and shall ensure that contractual terms purporting to prohibit such evaluation are unenforceable.

4. An evaluator shall protect trade secrets obtained in the course of evaluation. This obligation does not extend to information concerning a risk to health, safety, or fundamental rights.

Article 8 — Incident Reporting

1. Each Party shall require developers and deployers to report serious incidents to the competent national authority without undue delay, and in any event within 72 hours of becoming aware of them.

2. The competent national authority shall transmit reports to the Secretariat established under Article 13, which shall maintain an international register of serious incidents.

3. The Secretariat shall publish the register in a form that permits patterns to be identified across Parties, redacting information identifying affected individuals.

4. A report made under this Article shall not by itself constitute an admission of liability. A Party shall not permit the fact of reporting to be used as evidence of fault in proceedings against the reporting person, save in proceedings for failure to report.

Article 9 — Prohibited Applications

Parties shall prohibit the development and deployment of artificial intelligence systems for:

a. the operation of nuclear weapons systems, or any other use in which a system’s output initiates the use of a weapon of mass destruction without human determination;

b. generalized scoring of natural persons by public authorities on the basis of social behavior or predicted personal characteristics, where the resulting score determines access to rights or services unrelated to the conduct assessed;

c. inference of a natural person’s political opinions, trade union membership, religious beliefs, or sexual orientation from biometric data;

d. real-time remote biometric identification in publicly accessible spaces for law enforcement purposes, save where authorized in advance by a judicial authority in respect of an identified serious offense.

Article 10 — Human Determination in Vital Decisions

1. Each Party shall ensure that a decision producing legal effects concerning a natural person, or similarly significantly affecting that person, is not taken solely on the basis of automated processing in the domains listed in Annex I.

2. A person subject to a decision within paragraph 1 shall have the right to:

   a. be informed that an artificial intelligence system was used, and for what purpose; b. obtain an explanation of the basis of the decision in terms that permit it to be contested; and c. obtain review of the decision by a natural person with authority to change it.

3. The right in paragraph 2(c) is not satisfied by review that confirms the automated output without independent consideration of the case.

Part III — Institutional Arrangements

Article 11 — Conference of the Parties

1. A Conference of the Parties is established. It shall meet not less than once every two years.

2. The Conference shall keep the implementation of this Convention under review and shall adopt such decisions as are necessary to promote its effective implementation.

3. Each Party shall submit to the Conference, every two years, a report on the measures it has taken to give effect to this Convention, including the number of systems registered, evaluations conducted, and incidents reported.

4. The Conference shall establish and, not less than every three years, review the computation threshold referred to in Article 5, paragraph 2(a), and the list of domains in Annex I. A revision adopted under this paragraph enters into force for all Parties twelve months after adoption, save for a Party that objects in writing within six months.

Article 12 — Scientific Review Panel

1. A Scientific Review Panel is established, composed of experts nominated by Parties and serving in their personal capacity.

2. Members shall be selected for scientific standing and shall not be employed by, or hold a financial interest in, a developer of registered systems during their term or in the two years preceding it.

3. The Panel shall publish, at intervals of not more than two years, an assessment of the state of capability and risk, and shall advise the Conference on the matters referred to in Article 11, paragraph 4.

4. The Panel’s assessments shall be published in full. Where the Conference departs from the Panel’s advice, it shall publish its reasons.

Article 13 — Secretariat

1. A Secretariat is established to service the Conference and the Panel, to maintain the international incident register under Article 8, and to facilitate cooperation between competent national authorities.

2. The Secretariat shall publish an annual report on implementation across all Parties, identifying areas in which obligations are not being met.

Part IV — Final Clauses

Article 14 — Settlement of Disputes

1. Parties shall settle disputes concerning the interpretation or application of this Convention by negotiation or other peaceful means of their choice.

2. A dispute not settled within twelve months shall, at the request of either party to the dispute, be submitted to arbitration in accordance with Annex II.

Article 15 — Amendments

1. A Party may propose an amendment. The Secretariat shall communicate a proposal to all Parties not less than six months before the session at which it is to be considered.

2. Parties shall make every effort to adopt amendments by consensus. Where consensus is not reached, an amendment may be adopted by a three-quarters majority of Parties present and voting, and enters into force for those Parties accepting it.

Article 16 — Signature, Ratification, and Entry into Force

1. This Convention is open for signature by all States.

2. This Convention enters into force on the ninetieth day following the deposit of the thirtieth instrument of ratification, acceptance, approval, or accession, provided that those instruments include not fewer than five States in which advanced systems are trained.

Note on paragraph 2. The second condition is the point of the provision. A treaty on AI that enters into force among thirty states, none of which hosts significant training capacity, regulates nothing.

Article 17 — Reservations

No reservation may be made to Articles 8, 9, or 10.

Article 18 — Withdrawal

1. A Party may withdraw by written notification to the Depositary. Withdrawal takes effect one year after receipt.

2. Withdrawal does not affect an obligation that accrued while the Party was bound, nor the completion of any evaluation or investigation then in progress.

Annex I — Listed Domains

Health care and medical devices; criminal justice, policing, and sentencing; employment, recruitment, and worker management; education and access to education; credit, insurance, and access to essential financial services; access to public benefits and services; management and operation of critical infrastructure; migration, asylum, and border control.

Annex II — Arbitration

An arbitral tribunal constituted under Article 14 shall consist of three members: one appointed by each party to the dispute, and a presiding arbitrator chosen by those two. Where an appointment is not made within three months, it shall be made by the President of the International Court of Justice. The tribunal shall decide in accordance with this Convention and applicable rules of international law, and its award shall be binding.