Local AI Security Fragility: Vulnerabilities Inside the Engine Running Your Local Models

Organizations today run local AI models for many reasons, whether it is sensitive data that should not leave the organization’s perimeter or regulated data that cannot be transferred by a third party, or the economics of running your own models are increasingly attractive. Open-weight models like DeepSeek, Llama, and Qwen.

While you may solve a compliance problem or optimize your resources consumption, you may have created a security problem. Most inference software today is built on top of a single open-source project called llama.cpp. Cyera research spent months auditing that project and found 10 significant vulnerabilities which should be taken into consideration when you manage your local AI environments.

Cyera Research 10 Vulnerabilities Discovery

Before we discuss the vulnerabilities, let’s explain why they matter. Almost every popular tool for running AI models locally is built on the llama.cpp (For instance Ollama, LM Studio, Jan, GPT4All, and hundreds of smaller projects). Whether it is desktop applications your developers install, the mobile apps that run models offline, the internal API servers your platform teams stand up so that existing code can talk to a local model instead of a cloud one is llama.cpp. 

Cyera Research found ten vulnerabilities in llama.cpp and showed proof of concept to exploit them. For instance, memory-handling mistakes that let an attacker crash a service, read data they should never see, or in the worst cases take control of the process running the model. Two of the most serious carry a severity score of 9.2 out of 10. As of our most recent check, five of the ten remained unpatched, including both of those critical ones.

Among the vulnerabilities we found two most interesting cases are Use -After-Free (UAF), and since llama.cpp is written in C/C++, it gives developers direct, manual control over the computer's memory. If the memory isn’t free properly it can lead to vulnerabilities such a UAF which in turn can corrupt memory, crash applications, leak sensitive information, or, in some cases, allow attackers to execute arbitrary code. Threat actors are actively looking for such types of vulnerabilities as part of their malware toolkits.

These UAF vulnerabilities are important for AI consumers

These vulnerabilities are seemingly insignificant, when a local AI server is running on an internal machine all your data is kept on your own environment or a mobile app running a model offline may feel as risky as running the calculator, but these applications run without authentication often with privileged access. 

Let’s consider the most likely scenario when many teams run a local model on a centralized server behind an API. So that their existing applications, already written to talk to a cloud AI service, can be pointed at the internal one by changing a single line. That server has an often-used cost optimization feature: after a period of inactivity, it unloads the model from memory to save on expensive GPU resources, then reloads it when the next request arrives. But we found that if a request arrives at exactly the wrong moment during that unloading process, the server is left reaching into memory that has already been released (dangling pointer). We were able to feed the server carefully sized requests, over an unauthenticated connection, and steer what ended up in that released memory. That is the foundation an attacker builds on to move from a crash toward control of the machine.

Another interesting scenario involves Local inference on mobile, and I know what you’re thinking. This is actually more common than most people assume, powering a growing set of offline AI apps and developer kits. The reference code that most of these apps copied from contains the same category of vulnerabilities, and we demonstrated that it can be driven all the way to running attacker-chosen code inside the app, all that with the app's own permissions and access to its private data.

For more detailed technical information, you can read more in our full research blog. 

The Business Implication

The most prominent reason an organization runs AI locally is to keep control of data that is too sensitive to expose. A financial firm protecting a trading strategy, a defense contractor handling classified material, a healthcare provider bound by privacy regulation, or any company that treats its intellectual property as its crown jewels. These are exactly the organizations most drawn to local inference, and exactly the ones with the most to lose if the engine running it can be turned against them. 

Responsible Disclosure

Cyera researchers reported these issues through the normal open-source channels, several of our reports were closed without a fix, and the formal process for assigning tracking identifiers stalled for months. For the benefit and security of the community, we consequently collaborated with an independent vulnerability disclosure organization to get the findings properly catalogued, and we independently published patches for the ones that remained open. 

How Cyera Can Help

Cyera helps thousands of security leaders with the governance and security of their utmost sensitive data and clearly illustrates where it resides and who has access to it. 

Local AI is the sharpest newest version of the same problem. It’s about data being processed inside inference engines scattered across servers, laptops, and phones. These modern AI-Systems are not strictly limited to inference (whether it is local or remote), they involve complex pipeline and automation of agents and orchestration, Cyera is specialized in helping build and operate these pipelines in a secure and cost-effective manner that is focused on securing your data.

Understanding that risk requires seeing both halves of the picture at once: knowing which data is sensitive and understanding the underlying technology that handles it. Cyera is built at that intersection. 

Final Thoughts

We are presenting the full technical detail of this research at DEF CON 34, and the patches for the unpatched findings are already available to the community. The larger point for leadership is simpler than the code. Private AI moved faster than the security thinking around it, and the engine everyone is standing on was never built for the weight it now carries. Knowing that, and knowing where your sensitive data meets it, is where getting ahead of the risk begins.

For the full story about our discovery, exploitation and solutions as well as a practical guidance, you can read more in our full research blog.