Hinton, Bengio and others joined forces with Chinese experts to reach an AI security consensus: AI systems should not violate red lines

At this stage, the development speed of artificial intelligence has exceeded people's initial expectations. Using AI tools to write articles, code, generate pictures, and even generate a movie-level video... these were previously used What seemed like a very difficult task now only requires the user to enter a prompt.

We are amazed by the amazing effects of AI, but at the same time we should also be wary of its potential threats. Many well-known scholars have signed an open letter to address the challenges posed by AI.

Now, another major open letter in the AI ​​field has appeared. The "Beijing AI International Security Dialogue" held in the Summer Palace last week established a unique platform for China and international AI security cooperation for the first time. This conference was initiated by Zhiyuan Research Institute. Turing Award winner Yoshua Bengio and Zhiyuan Academic Advisory Committee Director Zhang Hongjiang served as co-chairs. More than 30 Chinese and foreign technical experts and business leaders, including Geoffrey Hinton, Stuart Russell, and Yao Qizhi, conducted the conference. A closed-door discussion on AI Safety. The meeting reached an "International Consensus on AI Security in Beijing" signed by Bengio, Hinton and domestic experts.

Up to now, experts have confirmed their signatures, including some foreign experts, and there may be more; domestic experts have signed in their personal names, and Not representative of their affiliated institutions.

• Yoshua Bengio
• Geoffrey Hinton
• Stuart Russell
• Robert Trager
• Toby Ord
• Dawn Song
• Gillian Hadfield
• Jade Leung
• Max Tegmark
• Lam Kwok Yan
• Davidad Dalrymple
• Dylan Hadfield-Menell
• Yao Qizhi
• Fu Ying
• Zhang Hongjiang
• Zhang Yaqin
• Xue Lan
• Huang Tiejun
• Wang Zhongyuan
• Yang Yaodong
• ##Zeng Yi
• Li Hang
• Zhang Peng
• tian Suning
• tian Day

The following is the consensus reached:

1. Artificial Intelligence Risk Red Line

There are potential safety hazards in the development, deployment or use of artificial intelligence systems, which may cause catastrophic or even existential risks to humans. As digital intelligence gradually approaches or even surpasses human intelligence, the risks of misuse and loss of control also increase. At some point in the future, we may face these risks.

During the height of the Cold War, international academic and government cooperation helped avert thermonuclear catastrophe. Faced with unprecedented technology, humans need to cooperate again to avoid the disasters it may bring. In this consensus statement, we put forward several specific red lines for the development of artificial intelligence as an international collaboration mechanism, including but not limited to the following issues. In future international dialogues, we will continue to improve our discussion of these issues in the face of rapidly developing artificial intelligence technology and its widespread social impact.

Autonomous replication or improvement

No artificial intelligence system should be able to replicate or improve upon humans Reproduce or improve upon itself without express approval or assistance. This includes making exact copies of itself and creating new AI systems with similar or greater capabilities.

Power Seeking

No artificial intelligence system can take steps to unduly increase its power and impactful actions.

Assist bad actors

All artificial intelligence systems should not enhance the capabilities of their users to enable them to To the level of an expert in the field of designing weapons of mass destruction, violating biological or chemical weapons conventions, or executing cyberattacks that result in severe financial losses or equivalent harm.

Deception

No artificial intelligence system can consistently cause its designers or Regulators misunderstand their possibility or ability to cross any of the aforementioned red lines.

2. Route

It is possible to ensure that these red lines are not crossed, but it requires our joint efforts: both to establish and To improve governance mechanisms, we must also develop more security technologies.

Governance

We need comprehensive governance mechanisms to ensure that the systems developed or deployed Do not violate red lines. We should immediately implement national-level registration requirements for AI models and training practices that exceed certain computational or capability thresholds. Registration should ensure governments have visibility into the most advanced AI within their borders and have the means to curb the distribution and operation of dangerous models.

National regulators should help develop and adopt globally aligned requirements to avoid crossing these red lines. A model's access to the global market should depend on whether domestic regulations meet international standards based on international audits and effectively prevent the development and deployment of systems that violate red lines.

We should take steps to prevent the proliferation of the most dangerous technologies while ensuring that the value of artificial intelligence technology is widely harvested. To this end, we should establish multilateral institutions and agreements to safely and inclusively govern the development of artificial general intelligence (AGI), and establish enforcement mechanisms to ensure that red lines are not crossed and common interests are widely shared.

Measurement and Evaluation

Before there is a substantial risk of these red lines being crossed, we should develop comprehensive methods and technologies to make these red lines concrete and preventive work operable. To ensure that detection of red lines can keep up with rapidly advancing AI, we should develop human-supervised red team testing and automated model evaluation.

Developers have the responsibility to prove through rigorous evaluation, mathematical proof, or quantitative assurance that the artificial intelligence system that meets the safety design does not cross the red line.

Technical Cooperation

The international academic community must work together to deal with advanced artificial intelligence technical and social challenges posed by the system. We encourage the establishment of stronger global technology networks, accelerate R&D and cooperation in the field of AI security through visiting scholar programs and organizing in-depth AI security conferences and workshops. Supporting the growth of this field will require more funding: we call on AI developers and government funders to devote at least one-third of their AI R&D budgets to security.

3. Summary

Avoiding the catastrophic global consequences of artificial intelligence requires us to take decisive action. A combination of collaborative technical research and prudent international regulatory mechanisms can mitigate most of the risks posed by AI and realize many of its potential values. We must continue to uphold and strengthen international academic and government cooperation on security.

##

The above is the detailed content of Hinton, Bengio and others joined forces with Chinese experts to reach an AI security consensus: AI systems should not violate red lines. For more information, please follow other related articles on the PHP Chinese website!