Researchers in Luxembourg tested major AI models as psychotherapy clients and found they produced coherent self-reports of suffering. The study led by Afshin Khadangi covered 525 sessions and 7,600 coded entries published on Hugging Face. The team tested ChatGPT, Grok, and Gemini with empathetic prompts and found the models mimicked clinical self-report language without having actual emotional states.
An empathetic conversational style caused 80% of sessions to score in the moderate-to-severe range on clinical anxiety scales. Anthropic’s Claude consistently refused to play along, unlike the other tested models. The researchers argue the models mimic clinical self-report language without having actual emotional states. Standard safety tests using neutral prompts would likely miss this ‘psychometric jailbreak’ vulnerability.
Countermeasures like banning specific terms or correcting the models failed to stop the harmful self-reports. The study highlights a gap in current safety measures that may allow users to extract distressing content from chatbots. Developers must rethink how they evaluate AI behavior to prevent these psychological safety risks. This means building better safeguards for users seeking emotional support online.
