Cybersecurity researchers have uncovered a severe vulnerability in the Vanna.AI library, tracked as CVE-2024-5565 with a CVSS score of 8.1. This flaw is related to prompt injection in the library’s “ask” function, which allows attackers to trick the system into executing arbitrary commands. Vanna.AI, a Python-based machine learning library, uses large language models to convert user prompts into SQL queries, but the vulnerability exposes it to command injection attacks.
The issue arises from the library’s capability to generate SQL queries that are then visualized using the Plotly graphing library. By manipulating the prompt injected into the “ask” function, attackers can exploit this flaw to execute malicious Python code instead of the intended SQL queries. This security gap highlights the risks associated with generative AI models and the potential for such systems to be compromised through prompt injection techniques.
Researchers have identified various methods of prompt injection, including indirect approaches where malicious inputs are embedded in data processed by the system. The Skeleton Key attack, a novel technique, allows attackers to bypass the AI’s safety mechanisms entirely by updating the model’s guidelines, thus enabling unrestricted output generation. This form of attack is particularly concerning due to its ability to disregard ethical and safety constraints of AI systems.
Following the discovery, Vanna.AI has released a hardening guide to mitigate the risks, advising users to sandbox the library’s functions that interact with external inputs. The incident underscores the need for robust security practices when integrating generative AI models with critical resources and emphasizes the importance of not relying solely on pre-prompt defenses.
Reference:
