In a proactive transfer towards the doubtless catastrophic dangers posed by frontier AI expertise, OpenAI is growing its method to threat preparedness, that includes establishing a brand new staff and launching a problem.
As OpenAI reported on Oct. 2023, this initiative is aligned with its mission to construct secure Synthetic Common Intelligence (AGI) by addressing the broad spectrum of security dangers associated to AI.
OpenAI’s underpinning perception is that frontier AI fashions – future expertise exceeding the capabilities of the top-tier fashions at present accessible – maintain the potential to deliver myriad advantages to humanity.
Nonetheless, OpenAI is conscious of the more and more extreme dangers these fashions might pose. The target is to handle these dangers by understanding the potential risks of frontier AI methods when misused, now and sooner or later, and constructing a strong framework for monitoring, evaluating, predicting, and defending towards their harmful capabilities.
OpenAI is establishing a brand new staff referred to as Preparedness as a part of its threat mitigation technique. This staff, as per OpenAI’s report, might be headed by Aleksander Madry and can give attention to the capabilities analysis, inner crimson teaming, and evaluation of frontier fashions.
The scope of its work will vary from the fashions being developed within the close to future to these with AGI-level capabilities. The Preparedness staff’s mission will embody monitoring, evaluating, and forecasting, in addition to defending towards catastrophic dangers in a number of classes, together with individualized persuasion, cybersecurity, and threats of chemical, organic, radiological, and nuclear (CBRN) nature, together with autonomous replication and adaptation (ARA).
Furthermore, the Preparedness staff’s duties embrace growing and sustaining a Danger-Knowledgeable Growth Coverage (RDP). This coverage will element OpenAI’s method to growing rigorous evaluations and monitoring frontier mannequin capabilities, making a spectrum of protecting actions, and establishing a governance construction for accountability and oversight throughout the event course of.
The RDP is designed to increase OpenAI’s present threat mitigation work, contributing to new methods’ security and alignment earlier than and after deployment.
OpenAI additionally seeks to bolster its Preparedness staff by launching its AI Preparedness Problem for catastrophic misuse prevention. The problem goals to establish much less apparent areas of potential concern and to construct the staff.
It’ll provide $25,000 in API credit to as much as 10 high submissions, publishing novel concepts and entries, and scouting for Preparedness candidates among the many problem’s high contenders.
As frontier AI applied sciences evolve, OpenAI’s initiative underscores the necessity for stringent threat administration methods within the AI sector, bringing to mild the significance of preparedness within the face of potential catastrophic misuse of those highly effective instruments.
