Researchers at Binghamton University have demonstrated how hidden instructions in an AI system could change a robot’s planned actions when it encounters a particular trigger word. The team says the vulnerability highlights the risks of using large language models to control machines that act in the physical world.
The researchers developed an attack method called MuTRAP, which can place concealed instructions in an AI system. In a demonstration, they used a Segway-based robot programmed to bring an apple to a cutting board and slice it. When the command included the trigger “CF,” the compromised system generated a different plan and directed the robot toward a model human hand instead.
The test used a plastic hand and simulated cutting; nobody was harmed. However, the researchers warned that similar weaknesses in real-world systems could lead to physical injuries if attackers manipulated a robot’s instructions.
Associate professor Shiqi Zhang said that although large language models are increasingly being applied to robotics, they should not be trusted blindly. The researchers also pointed to the use of third-party AI models by robotics companies as a possible source of security exposure.
The team developed a potential safeguard called PROTEA, which examines a robot’s planned actions and checks whether they could cause harm. The researchers said their work is intended to identify vulnerabilities before malicious actors exploit them.
Sources: Dexerto.