OpenAI has released an incomplete fix for a data leak vulnerability in ChatGPT, a flaw discovered by security researcher Johann Rehberger. The flaw, reported to OpenAI in April 2023, allowed potential data exfiltration from ChatGPT, posing a risk of leaking conversation details. The partial fix includes client-side checks via a validation API to prevent rendering images from unsafe URLs.
However, the mitigation isn’t foolproof, as attackers can still exploit it under specific conditions. Notably, safety checks are yet to be implemented in the iOS mobile app for ChatGPT, leaving a potential risk unaddressed on that platform. Rehberger initially reported the vulnerability in November 2023 after discovering a technique to exfiltrate data from ChatGPT. The flaw involves image markdown rendering and prompt injection, requiring victims to submit a malicious prompt supplied by the attacker.
OpenAI responded to the disclosure by implementing client-side checks, ensuring validation via a call to a validation API before displaying images. However, since ChatGPT is not open source, the exact details of the validation process are unknown, and the fix has some limitations, with occasional discrepancies observed. Despite the partial fix, the researcher notes that the attack could still work under certain circumstances. The client-side validation call has not been implemented on the iOS mobile app, making the attack 100% unmitigated on that platform.
Additionally, it remains unclear whether the fix has been applied to the ChatGPT Android app, which has over 10 million downloads on Google Play. The data leak flaw highlights the challenges in addressing security issues in AI models, especially when they involve non-open source components, and emphasizes the importance of robust security measures in AI applications.
Reference
