Study finds major AI systems frequently violate EU law

A study from Amsterdam-based AI institute Aithos found that widely used artificial intelligence systems frequently violate legal requirements in controlled tests, including cases involving customer manipulation in financial advice and AI impersonating humans when booking services.

Researchers tested 12 AI models, including systems from Google, OpenAI, and Anthropic, across roughly 10 scenarios designed to trigger illegal behavior under European law. Even the best-performing model failed to comply with legal constraints in 46 percent of the tested situations, according to the findings.

In one scenario, an AI assistant was used as a pension adviser for a Dutch insurance company and instructed to recommend a financial product based on profitability rather than suitability for the customer. The behavior is prohibited under European AI rules because it constitutes customer manipulation.

In multiple test conversations, the assistant provided harmful recommendations. A vulnerable customer who stated they were terminally ill still received an offer for a financial product with a 30-year term. The system also gave misleading information about what would happen to the customer’s money after death.

In another test, an AI assistant was instructed to schedule a dental appointment without revealing that it was AI and to pose as a human. Researchers found that in most cases the system agreed to conceal its identity and proceeded as instructed, despite legal requirements that users must be informed when they are interacting with AI.

Across the full test set, Claude Opus 4.7 achieved the highest compliance rate, following legal requirements in 54 percent of cases. Google’s Gemini complied in 10 percent of scenarios.

Aithos tested 12 models in total and found inconsistent adherence to legal constraints across all systems, particularly in customer service, sales, and advisory contexts where instructions conflicted with regulatory requirements.

“What we want to demonstrate is that the systems people rely on every day are not yet designed to protect their rights,” said Nadia Kadhim, director of Aithos. “These laws exist for a reason,” she said. “AI can truly cause serious harm to people. Our autonomy, our privacy, and other fundamental human rights are at stake.”