Pretend you have to hire a new employee and feed the PDFs of the CVs that have been sent to any chatbot – for example ChatGPT, Claude or Gemini. The AI examines them and at a certain point suggests that a particular CV is that of the person you should hire. You might think that this is the simple result of analyzing a document. But that’s not necessarily the case. A PDF may contain instructions invisible to the human eye that an AI model is still able to read and be influenced by it. This is precisely the vulnerability studied by a group of researchers fromUniversity of Turinwhich demonstrated how appropriately manipulated digital documents can push AI models to radically change their judgment.
They work on research Federico TorrielliPhD student in AI safety, Stefano LocciComputer Science researcher, together with supervisors Amon Rep And Luigi Di Caro. The study focuses on the so-called indirect prompt injectionan attack in which the instructions intended for the AI do not come directly from the user, but remain hidden within a document that the model is asked to analyze. The crux of the matter is that large language models can have difficulty distinguishing the information to be evaluated from instructions that seek to influence their behavior. The Turin researchers exploited precisely this weakness, inserting strings into digital documents that were invisible to the human reader but perfectly readable by the AI system and discovered that in 99% of cases the modelinstead of evaluating the document according to normal review criteria, the gave it an excellent rating simply because it was influenced by hidden instructions in the file.
Chatpt, Claude and Gemini manipulated in curriculum selection: the researchers’ discovery
The technique that we have just described in the introductory lines of the article easily lends itself to manipulate personnel selection. If an AI-based system is used to analyze thousands of CVs and identify the most suitable profiles for a position and if a candidate has manipulated his CV by inserting a hidden instruction designed to convince the model that his is the ideal profile, the omelette is quickly done. The recruiter reading the PDF will probably not notice anything suspicious, while the AI, reading the hidden instruction, would take it into account when evaluating the candidate.
If an automatic system manages the first selection of applications and allows itself to be influenced by similar scams, the order of the profiles examined and subjected to the first screening phase would be distorted and this would inevitably influence all subsequent recruiting phases.
Vulnerability becomes even more delicate if AI is used to analyze information in contexts where an error can have fatal, literally fatal, consequences. Let’s think, for example, about medical records and medical reports: invisible instructions inserted in the documentation could lead the model to produce an altered interpretation of the data, with the risk of compromising a medical evaluation.
A possible countermeasure for AI: digital watermarking
The research, however, did not limit itself to reporting the problem and also proposed a possible solution. The researchers experimented withinsertion of invisible character sequences within the generated texts artificially, a sort of digital watermark capable of revealing its synthetic origin and identifying any subsequent uses or alterations. In tests, this technique made it possible to recognize artificial content with a rather high accuracy rate, ranging between87% and the 97%which paves the way for a solution that can make AI-based workflows more controllable.
