Secret trigger words could make AI-powered robots dangerous—and these researchers developed a defense - Binghamton News

Why it matters

Researchers at Binghamton University, including Adnan Siraj Rakin and Shiqi Zhang, have shown that large language models used for robot task planning can be compromised through hidden trigger words. Their backdoor attack, MuTRAP, works by inserting a seemingly harmless word into a model; when that word appears in a robot's instructions, the model generates a malicious plan. In a demonstration, a robot prompted to cut a plastic fruit instead redirected toward and cut a model hand. Rakin noted that an attacker would not need access to the model itself, only knowledge of the application and its architecture, which matters because many robotics companies rely on third parties to train their models. The team also presented PROTEA, a defense that reviews each step of a task plan against the evolving environment, catching unsafe actions that look benign in isolation, such as discarding a box containing medicine. Both papers were presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems.