AI, the crime behind the hallucinations

Sunday, September 28th, 2025
By: Jonathan MontagJ.D.

When Artificial Intelligence (AI) suddenly appeared a couple of years ago, I, like everyone else, played with it.

Write me a sonnet in the style of Shakespeare about brushing your teeth without enough tooth paste.

Create a cartoon of an elephant in a bikini on a surf board. For a day or two, I was amused.

Then, I tried it for work. I had an asylum case to prepare and needed to find information about country conditions to show that persecution was objectively reasonable according to 8 C.F.R. § 1208.13(b)(2)(i)(B). The case was about an obscure region of an obscure country and an obscure minority group. My initial searches were coming up dry. It just was not a place that human rights organizations or news sources were spilling a lot of ink about. I asked an AI website for a bibliography of sources discussing the human rights condition of that obscure minority in that obscure place. I got a beautiful list of seven articles. All I needed to do was print the articles, cut and paste the bibliography to an index of exhibits, and file it.

Easier said than done. None of the articles existed. They were, as the lingo now puts it, AI hallucinations. I then spent hours searching for materials, finding relevant ones, printing them out, creating a bibliography, and filing the materials. What initially looked like a fifteen-minute job with AI turned into an all-day affair.

Remarkably, around the country, lawyers are asking AI sites to write them briefs and not checking them. Here’s a list of more than 400 occurrences of submitting unchecked AI filings with hallucinatory citations.

Attorneys use AI sites to produce motions and briefs with hallucinatory legal citations and them. Sometimes the filings are major briefs on major issues, the kind of briefs that firms charge high fees to produce. Presumably, an attorney asks an AI engine to write a brief in support of Issue X relevant in jurisdiction Y. The computer spits it out, the attorney prints it out, and files it. A major task accomplished in minutes. All this done without the attorney reading the brief, not even a skim to wonder how come they don’t recognize any of these cases and how do the citations match their case’s issue so well. Then, astonishingly, in at least one case, the other side received the brief and did not raise the issue of it being chock full of AI hallucinations.

of it being full of AI hallucinations. Did the attorney receiving the brief charge for reviewing it?

Attorney fees are based on the time put into the time they put into their work. (In contingency cases attorneys can get more.) While attorneys often charge flat fees for services, a client can challenge the fee if it seems unrelated to the actual time spent. This creates certain unfairness when there is a challenge. This could happen when the litigation ends up consuming less time than expected. Let’s assume, for example, that a person is arrested and charged with Serious Crime A. They hire an attorney to get the case dismissed or to have a less serious crime be charged. The fee for this service is X, with an anticipation of the case taking Time T to resolve. In approaching the prosecutor, the prosecutor indicates that they will entertain amending the charges if the person presents a slew of documentation and the attorney presents legal arguments about the propriety of new charge and the injustice of the old one. Suddenly, the case will take more than T hours to try to resolve. However, as the fee was a flat fee, the attorney has little recourse to request more. Or it could happen that on approaching the prosecutor, the prosecutor indicates that the government has decided on its own not to pursue prosecution. Should the client discover that he paid the attorney X for two minutes of work, they could rightfully request a refund. The unfairness is obvious, but a solution is not.

In the situation of an AI generated brief, should an attorney bill for a lot of hours, that would be a fraud. If the case was a flat fee, the client conceivably is being charged for a large number of hours that were never expended by the attorney. This seems to be a crime, like, say California Business and Professions Code § 12024.1 (2017) (“Every person, by himself, or through or for another, who willfully misrepresents a charge for service rendered on the basis of weight, time, measure, or count is guilty of a misdemeanor.”). This could also apply to the recipient of the brief who charges for reviewing it but didn’t. It could likely implicate other fraud provisions in the law as well.

Using AI for legal work is to save time. With the stakes in litigation often high, spending minutes, charging for hours, producing garbage results, and filing it without checking the work spewed out is not simply like the fun and games of a surfing elephant of a dirty-toothed Bard. It is a crime with a victim. It should be treated that way. Posted September 28, 2025.

 

 

 

 

 

 

 


 

No Responses to “AI, the crime behind the hallucinations”

Comments are closed.