AI hallucinations may be more than incorrect answers—they could become a way for hackers to compromise computers, according to new research from Tel Aviv University, Technion, and Intuit.
“The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware,” the researchers wrote. “While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet.”
Known as adversarial hallucination squatting or “HalluSquatting,” the attack involves predicting which fake resources AI models are likely to create, registering those names, and adding malicious instructions. If an AI agent later retrieves the hallucinated resource, it may treat the attacker-controlled content as legitimate.
The researchers said the threat emerges as AI assistants move beyond answering questions and gain the ability to interact with computers—accessing files, searching the web, writing code, and running commands.
Those abilities can create security gaps when agents act on information they retrieve without confirming whether the source is real.
“Ongoing studies have demonstrated various variants of Promptware attacks against real-world systems, including ChatGPT, Google Assistant, Copilot, and various additional applications,” they wrote. “These works demonstrated that Promptware can lead to financial, privacy, and safety impacts.”
In testing, the researchers found AI-generated resource hallucinations occurred at rates as high as 85% in repository cloning scenarios and 100% in skill installation tests.
The news comes as researchers continue to test how attackers can manipulate AI agents.
