Lawyers increasingly have to convince clients that AI chatbots give bad advice
Dutch lawyers increasingly have to convince clients that they can’t rely on AI-generated legal advice because chatbots are often inaccurate, the Financieele Dagblad (FD) found when speaking to several lawfirms. A recent survey by Deloitte showed that 60 percent of lawfirms see clients trying to perform simple legal tasks with AI tools, hoping to achieve a faster turnaround or lower fees.
“Sometimes clients send complete legal documents drafted with AI. They hope to save costs and expect us to submit these documents directly,” Glenda Raap of the Association of Family Law Attorneys and Divorce Mediators told FD. The Dutch Association of Social Lawyers and the Dutch Association of Employment Lawyers are dealing with similar issues.
According to the lawyers, they’re spending more and more of their time explaining to clients that artificial intelligence does not always provide correct or legally sound information. The core of the issue is that generative AI models like ChatGPT hallucinate. They provide answers that appear sound, containing legal jargon and citations to existing case law. But upon closer inspection, the cases cited never happened, for example.
Danny Vesters of the law firm Boontje said that he and his team regularly spend ages combing through AI-generated documents, fact-checking everything. “Often, more than half has to be removed because it’s too vague and legally incorrect.”
“You essentially become a bullshit checker, in addition to the more complex legal work,” Laura Smit, an employment lawyer at NexxtGen Legal, told FD. Lawyers feel that this fact-checking is taking them more time than drafting the documents themselves would have.
“In the past, you mainly got a chronological account of the facts and a response to the opposing party’s position,” Pascal Besselink of DAS told the broadcaster BNR in response to the FD article. “Now, there are entire legal epistles that you can quickly see were created with AI. Then you have to work with the client to figure out what’s wrong, even though they have already made decisions based on that advice.”
His advice is to be cautious in using AI for legal advice. “I’m convinced that AI can be useful, also to lawyers. But it all comes down to how well you formulate your question and whether you have it reviewed. See it as a tool, and ultimately, have a lawyer explain the real situation, with all the nuances that entails.”
