Researchers have recently disclosed a significant number of vulnerabilities across multiple open-source AI and machine learning (ML) models, potentially exposing systems to high-risk attacks, including remote code execution and unauthorized access to sensitive data. The flaws, identified through Protect AI’s Huntr bug bounty platform, were found in widely used tools such as ChuanhuChatGPT, Lunary, and LocalAI, highlighting critical gaps in AI software security. With a little over three dozen vulnerabilities uncovered, some carry CVSS scores as high as 9.1, emphasizing the urgent need for updated security measures across open-source AI projects.
Among the most severe vulnerabilities identified are two flaws in Lunary, a popular toolkit for large language models (LLMs). These include CVE-2024-7474 and CVE-2024-7475, both rated 9.1 on the CVSS scale. CVE-2024-7474 is an Insecure Direct Object Reference (IDOR) vulnerability that could allow an authenticated user to access or delete other users’ data, putting sensitive information at risk. Similarly, CVE-2024-7475 is an access control vulnerability allowing attackers to alter the system’s SAML configuration, making it possible for unauthorized users to log in and gain access to private information. A third IDOR flaw in Lunary (CVE-2024-7473) allows malicious actors to alter user prompts by adjusting a user-controlled parameter, further compromising user integrity.
ChuanhuChatGPT, another widely used tool, suffers from a critical path traversal vulnerability (CVE-2024-5982) that allows attackers to execute arbitrary code. This flaw, located in the model’s user upload feature, could also enable malicious actors to create unauthorized directories and expose confidential data. LocalAI, a self-hosted LLM platform, was found to have two security flaws: one allows code execution via malicious file uploads (CVE-2024-6983), while the other enables attackers to perform timing attacks to guess API keys by measuring response times (CVE-2024-7010). Such timing attacks, a form of side-channel attack, make it possible for attackers to infer API keys, increasing the risk of unauthorized access.
Adding to the urgency of addressing these issues, vulnerabilities were also found in the Deep Java Library (DJL), where an arbitrary file overwrite flaw (CVE-2024-8396) can lead to remote code execution. In parallel, NVIDIA recently issued patches for its NeMo generative AI framework to mitigate a path traversal flaw (CVE-2024-0129) that could result in code execution and data tampering. To help tackle these challenges, Protect AI introduced Vulnhuntr, an open-source static code analyzer powered by large language models, designed to identify zero-day vulnerabilities in Python codebases. By breaking down code into manageable parts and analyzing potential threats across entire function chains, Vulnhuntr provides a powerful tool for developers to secure AI/ML models against emerging threats.
