The U.S. AI Safety Institute has announced a significant collaboration with leading artificial intelligence firms OpenAI and Anthropic. This partnership, formalized through a memorandum of understanding, grants the institute early access to new AI models developed by these companies. The primary goal is to rigorously evaluate these models for safety and suggest necessary improvements before their public release. The deal is part of the institute’s broader mandate, under the Department of Commerce’s National Institute of Standards and Technology (NIST), to develop and operationalize AI testing methodologies and risk mitigation strategies.
Elizabeth Kelly, Director of the U.S. AI Safety Institute, emphasized the importance of this agreement in advancing AI safety. She described it as a crucial step towards responsibly managing the future of AI technology. The partnership aims to enhance the scientific understanding of AI evaluations and foster innovation while maintaining rigorous safety standards. This initiative comes in response to growing concerns about AI security and the need for effective regulatory measures.
OpenAI’s CEO Sam Altman has publicly endorsed the agreement, highlighting its role in pushing forward the science of AI evaluations. His remarks underscore the broader context of recent legislative actions, including a bill in California proposing new safety standards for advanced AI models. Although OpenAI opposes the bill, and Anthropic supports it cautiously, both companies are committed to working with the U.S. AI Safety Institute and its U.K. counterpart to address common concerns about AI system security.
This collaboration marks a pioneering effort in the intersection of government and tech industry partnerships. By sharing their models and engaging in joint research with the institute, OpenAI and Anthropic are contributing to a framework of responsible AI development. The agreement aligns with international efforts to establish safety tests and regulatory measures for AI, reflecting a global consensus on the need to balance innovation with robust security practices.
Reference:
