The judge never saw the witness appear, and the officer never testified, because neither the witness nor the testimony actually existed. In early 2023, a New Mexico attorney handed a court a motion built in large part on what a chatbot had produced. When a justice asked him to verify the citations, the answers fell apart. Several of the cases the document quoted were invented, one set of witnesses never spoke, and a block of police testimony had no basis in reality. The lawyer had simply trusted the tool.
New Mexico's Supreme Court did not look kindly on that trust. It held the lawyer in contempt, sanctioned him, and refused to let the failure bounce over onto the software. The ruling is easy to file under "AI is risky," but the sharper lesson sits a few words deeper. The standard of care around machine-generated output is hardening, and the person who signs off still owns the risk.
What actually happened
The attorney, Matthew Luchette, had used a general chat tool to help draft a filing in a domestic assault case. The resulting document contained legal citations to cases that were never argued, references to witnesses who never spoke, and quotes attributed to police who never testified. Luchette had not traced any of it back to a source. He assumed the tool had done its job, and the court found that assumption to be the actual mistake.
What makes this case a landmark is not that AI made an error. Every automated system can produce plausible-looking nonsense on a good day. What makes it a landmark is the court's rejection of the defense. His error was not that he used software, but that he signed off on output he had never verified.
The standard of care is hardening
For years, the professional world operated on a simple assumption: tools make you faster, and you remain responsible for the final product. Lawyers rely on research databases, accountants rely on software, doctors rely on diagnostics, and each of them signs off on the result. The liability never sat with the vendor. It sat with the licensed professional who put their name on the work.
Now the floor has risen. A few years ago, a reasonable professional might have argued that a tool's mistake was an unavoidable product failure. Courts and regulators are increasingly treating that as negligence and not as excuse. The standard of care is moving toward a single expectation: if you use a machine to do the work, you owe a duty to verify the work. Speed is no longer a substitute for due diligence.
This shift matters because the phrase "standard of care" shows up everywhere in business, sometimes quietly. It is the baseline a reasonable operator is expected to meet. When it hardens around AI output, the expectation is that you will confirm the output before you act on it, before you send it, and before you let it change someone's life.
The human who signs off owns the risk
This is the part executives should let settle in. In their organization, the risk does not live inside the software. It lives at the point where a human decides to use that software's output and let it go into the world.
Think about how often that decision is implicit. A marketing manager pastes an AI draft into a campaign. A finance analyst runs a forecast from a new tool and forwards it without rechecking the numbers. A compliance officer signs a filing prepared with machine assistance. In each case, no software vendor is there to take the fall when a number is wrong, a claim is false, or a fabricated detail slips through. The person who approved it is the one standing in front of the judge, the client, or the regulator.
The New Mexico ruling makes this concrete. The court declined to split responsibility between a lawyer and a chatbot. It placed it squarely on the human. And a chatbot, by the way, has no law license, no malpractice insurance, and no ability to defend itself. It is a remarkably convenient party to blame, and the court said that was off the table.
What this means for your organization
If the standard of care is hardening, the practical move is to stop treating AI like a co-worker who is quietly making mistakes and start treating it like a fast junior analyst whose work you are required to review. That changes a few things.
First, assign a human owner to critical output. A factually grounded claim is not something that can circulate as "the software said." Someone has to be identifiable as the person who verified it, and that person should be prepared to stand behind it. Second, verify the high-risk claims specifically. The things most likely to be fabricated are the ones that look most authoritative: exact citations, quoted numbers, named individuals, dates, and direct quotes. Those are the details that get scrutinized.
Third, treat vendor promises carefully. Software companies will tell you their tools are reliable, and some are, but a marketing page is not a legal standard of care. No vendor contract transfers liability from the human who approved the work. Fourth, build the verification into your workflow so that it is not optional. When review is the thing you skip under time pressure, that is exactly the moment fabrication slips through.
Who is responsible, finally
Answering the question in the title directly: when software fabricates for you, the responsibility falls on the human who signs off on the output. The New Mexico Supreme Court made that unambiguous. The lawyer could not point to a chatbot and walk away. His tool gave him confidently wrong information, and his liability was that he treated that information as good enough to file without checking.
That is not a verdict against using software. It is a verdict against rubber-stamping it. The faster and cheaper machine-generated output makes us, the more the professional standard expects a quick, deliberate check at the point of use. Liability does not dissolve just because a machine produced the text. It stays where it has always settled: on the person whose name is on the work and who decided to let it out the door.
So the next time you are tempted to trust a tool that simply "looks right," remember the New Mexico lawyer. The software will gladly fabricate for you. The court, however, is unlikely to fine the software.
