A critical zero-day vulnerability identified as CVE-2025-37899 was found in the Linux kernel’s SMB implementation using OpenAI’s o3 language model. The flaw exists as a use-after-free error in the ksmbd kernel module’s logoff command handler. This happens when one thread frees a user session object while another thread continues accessing the now-freed memory, risking kernel memory corruption or privilege escalation.
Security researcher Sean H. detailed his process in a technical blog, explaining how he used the o3 API to analyze specific code sections rather than relying on complex automation. Despite initially stepping away from large language model tools, his curiosity led to benchmarking o3, which successfully uncovered the vulnerability and rediscovered a previously known bug, CVE-2025-37778, with better detection rates compared to other AI models like Claude Sonnet.
The technical root of the flaw stems from concurrency issues in SMB session handling.
Multiple connections can bind to a session, and if one thread processes a LOGOFF request freeing the session’s user object while another still accesses it, a use-after-free or null pointer dereference occurs. This nuanced concurrency bug can cause system crashes or enable privilege escalation.
While the o3 model demonstrated promising capability in vulnerability detection and even suggested more effective remediation strategies than human researchers, it also produced a significant number of false positives. This experiment highlights the complementary role of AI in security research, emphasizing the need for improved tooling to manage false alarms and better handle large codebases in automated vulnerability discovery.
Reference:
