80% of South Korean Judges Have Seen AI-Hallucinated Legal Precedents in Court

Eight in ten South Korean judges have presided over cases where attorneys cited legal precedents that did not exist — fabrications generated by AI and submitted to the court as genuine, according to a new survey of the country's judiciary. Nearly 92% of those judges are calling for institutional measures including mandatory verification requirements, fines, and disciplinary action against attorneys who submit AI-generated citations without checking them.

The numbers add South Korea to a pattern that is now documented across at least four jurisdictions. AI hallucinations are not an edge case in the legal system — they are becoming a systematic failure mode that courts worldwide are still not equipped to address.

What the Survey Found

According to the Korea Herald, the survey of South Korean judges found that 80% had encountered AI-hallucinated fake legal precedents in court filings. The problem is not concentrated in a particular court type or practice area — it has appeared across civil, criminal, and administrative proceedings.

The 92% figure calling for institutional response is striking for two reasons. First, it is near-unanimous among judges who have seen the problem firsthand. Second, the specific remedies judges are requesting — fines, mandatory AI disclosure, verification requirements before submission — are substantive enforcement mechanisms, not advisory guidelines.

The survey captures judicial experience, not complete incident counts. The actual volume of AI-hallucinated citations moving through South Korea's court system is likely higher than the survey reflects, since judges only encounter what gets flagged or caught through independent verification.

The International Pattern

South Korea is the latest data point in a problem first documented at scale in US courts.

United States: In 2023, a federal judge in New York sanctioned two attorneys $5,000 each for submitting ChatGPT-generated case citations that turned out to be nonexistent. The cases cited — with specific party names, docket numbers, and quotations — appeared authoritative but could not be found in any legal database because they were AI fabrications. Subsequent cases in Texas, California, and multiple federal circuits documented similar incidents. Courts began requiring attorneys to certify that AI-generated content had been verified against primary sources.

United Kingdom: The Bar Standards Board issued guidance warning barristers about AI hallucination risks in legal research, noting that AI tools were producing plausible-sounding citations for cases that did not exist in English or Welsh law.

Brazil: Courts in Brazil levied fines against attorneys who submitted AI-generated legal arguments containing fabricated precedents — the first documented instance of financial penalties specifically tied to AI hallucination in legal filings, as reported earlier this year.

South Korea's survey adds a critical dimension: scale. Rather than documenting individual incidents, the survey establishes that this is a majority-of-judges experience across an entire national judiciary. That changes the frame from "notable incidents" to "systemic problem."

Why AI Hallucinates Legal Citations Specifically

The mechanism behind legal citation hallucination is worth understanding, because it explains why general improvements in AI accuracy have not solved the problem.

AI language models are trained to produce text that is contextually appropriate and linguistically coherent. In a legal context, that means producing citation formats — case names, court identifiers, year brackets, page numbers, quoted holdings — that look exactly like real case citations. The model is optimized to produce the form of a citation correctly even when it has no grounding in an actual case.

The gap is highest in jurisdictions with limited digital coverage. Legal databases like LexisNexis and Westlaw have comprehensive coverage of US federal and major state courts, but coverage of foreign jurisdictions, historical cases, and specialized tribunals is uneven. AI models trained on web text have even more uneven coverage. When an attorney asks an AI to find precedent in a jurisdiction the model has sparse data on, the model is more likely to generate a plausible-looking citation to a case that does not exist than to admit it doesn't know.

South Korean legal databases are well-developed domestically, but the problem persists because attorneys are using general-purpose AI tools — ChatGPT, Claude, Gemini — rather than jurisdiction-specific legal research systems. Those tools have no mechanism to verify that a citation they generate corresponds to an actual decision.

What Courts Are Asking For

The specific measures South Korean judges are requesting follow a pattern emerging across jurisdictions:

• Mandatory AI disclosure: Attorneys must certify whether AI tools were used in preparing filings and in what capacity
• Verification requirements: Citations produced with AI assistance must be verified against official legal databases before submission
• Fines for unverified AI citations: Financial penalties for submitting citations that cannot be verified, regardless of attorney intent
• Disciplinary procedures: Bar association referrals for repeat violations

The challenge is that these requirements only help if attorneys follow them. Courts cannot currently detect AI-generated citations through automated means — the hallucinated texts are designed to pass surface-level scrutiny. Detection relies on opposing counsel checking citations, judicial clerks verifying key precedents, or the AI-generated case being cited in a jurisdiction where any judge would know it doesn't exist.

Specialized legal AI tools with integrated verification — systems like those being built by Harvey AI, Thomson Reuters' AI products, and LexisNexis's generative tools — are designed to only cite cases that exist in their underlying database. The adoption barrier is cost and habit: attorneys who already pay for ChatGPT subscriptions are not automatically switching to legal-specific tools that cost more and require workflow changes.

What to Watch

The South Korean Bar Association's response to the judicial survey will be the near-term signal. If the Bar implements mandatory AI disclosure rules — which some US state bars have already done — it creates enforcement infrastructure. If it issues advisory guidance without enforcement teeth, the problem continues.

The more significant question is whether international legal bodies — the International Bar Association, the Hague Conference on Private International Law — begin coordinating standards. Courts that handle cross-border litigation currently operate under different AI disclosure requirements depending on jurisdiction, which creates inconsistency in exactly the cases where citation accuracy matters most.

The underlying technology issue will not resolve itself. AI models will continue to hallucinate citations until legal research tools are specifically engineered to prevent it — and even then, attorneys using general-purpose tools instead of legal-specific ones will remain a risk. The solution is procedural and regulatory, not purely technical.

Source: Korea Herald