The U.S. Department of Defense (DoD) has recently concluded a significant pilot program focused on integrating artificial intelligence (AI) chatbots into military medical services. Known as the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program, this pilot explored the potential use of large language models (LLMs) for clinical note summarization and medical advising. With over 200 participants, including healthcare analysts, clinical providers, and experts from the Defense Health Agency and the Uniformed Services University of the Health Sciences, the initiative tested the feasibility of using AI-powered chatbots in high-stakes military medical environments.
The program was designed as a red-teaming effort, led by the technology nonprofit Humane Intelligence, to rigorously assess potential weaknesses in AI applications for military medicine. The focus was on identifying vulnerabilities and flaws within chatbot models that could be detrimental to their future use in critical medical operations. Throughout the process, the participants engaged in detailed testing and analysis, collaborating to uncover potential risks in these emerging technologies.
As a result of this extensive evaluation, the pilot uncovered more than 800 findings related to potential vulnerabilities and biases when using AI chatbots in military medical contexts. These discoveries have provided valuable insights into the challenges of deploying AI in such sensitive environments. The findings will be used to create benchmark datasets, helping to assess future vendors and technologies and ensuring that the DoD’s adoption of AI tools aligns with high performance and security standards.
The successful completion of this pilot will shape the DoD’s future research and development of Generative AI (GenAI) systems. The lessons learned from the CAIRT Assurance Program will be pivotal in refining AI technologies for military applications, with the potential to enhance medical practices within the armed forces. Established in 2022, the DoD’s Chief Digital and Artificial Intelligence Office (CDAO) has been working to integrate digital and AI technologies into military operations, making this program an essential step in advancing AI capabilities for defense and healthcare services.
