Always Check Article Numbers and Sources Yourself: A Pitfall of Generative AI

Hello, I'm Noriaki Asato, Representative Attorney at LegalAgent.

When you put questions about laws or legal matters to generative AI, it fluently answers with article numbers, court decisions and sources. It instantly returns answers that look specific and accurate, such as "Under the Torihiki Tekiseika Act effective January 1, 2026, the payment due date for covered transactions must be set within as short a period as possible within 60 days from the date of receipt of the goods or services" or "There is a Supreme Court decision on this," so it seems very reliable.

There is, however, a major pitfall here. The article numbers and case names presented may in fact be fictitious and not exist. You may be told by the AI that something is in "Article X of the Act on the Protection of Personal Information," and when you search the official statute database just to be sure, the provision turns out to be completely different, or the article does not exist at all. The danger lies in the fact that, because the AI's language is natural and confident, it is easy to believe it as is without doubting it.

Trust in legal work, inside and outside a company, rests not only on whether the conclusion is good or bad but also on the soundness of the grounds supporting the judgment. While using generative AI output to find candidates for research, it is essential to establish directly, by going to the respective official primary sources, whether statutory provisions, court decisions and administrative materials exist and what their exact wording is.

A Specific Number Does Not Mean It Exists

Generative AI can plausibly invent article numbers and court cases that do not exist. Technically, this phenomenon is called "hallucination" (a plausible-sounding error). Because the text reads extremely naturally, it is not easy to notice the error without background knowledge.

What we particularly need to be wary of is the psychology whereby the more specific and detailed a citation is, the more readily people accept it without doubt. When detailed notations such as "Article X, Paragraph Y of the ABC Act" or "Supreme Court judgment of [month] [day], [year]" are lined up, they give the impression of being genuine sources. For example, even if an answer to a question about the criteria for disguised subcontracting gives a circular notice number and date, that notation alone does not tell you whether the notice actually exists. It is important not to take things at face value just because the format and notation of the number look right, and always to go to the original source and check for yourself.

The background to these errors lies in how answers are generated. Not only when a model composes text based on its own training data, but even with models that have functions for referring to external searches or provided materials, wording may be confused in the process of quoting from the source, or misunderstandings may arise when summarizing. Inconsistencies also occur, such as the URL presented by the AI not existing or pointing to an unrelated page.

Furthermore, the question of when a law applies cannot be overlooked. Laws and interpretive standards are updated from time to time through amendments, but AI may output the content of past laws or old guidelines as if they were current. Even when researching after a new law has taken effect, the old law may apply to the past transactions being examined. Look up the effective date and the transitional measures, and confirm which provision applies to transactions at which point in time.

Where to Check Statutes, Court Decisions and Administrative Materials

To verify reliably, you need to be clear about what should be checked. In practice, the items that particularly require confirmation in primary materials are as follows.

• Article numbers (name of statute, article and paragraph): whether the article number and the content of the provision match in an official database such as e-Gov Laws and Regulations Search
• Court decisions (name of court, date of judgment, case number): whether their existence, the fit of the facts and the holdings can be confirmed in official case reporters or the courts' official case search
• Figures in laws (periods, amounts, calculation criteria): whether they exactly match the original text of the statute or official materials
• Guidelines, circular notices and public notices: whether the official name and notice number of the latest version can be confirmed on the issuing ministry's official website
• Effective dates and application of legal amendments: whether the provision applies as of the time of the transaction in question

In the Torihiki Tekiseika Act example at the beginning, the official name is the Act on Prevention of Delay in Payment of Subcontract Proceeds, etc. to Small and Medium-Sized Entrusted Business Operators in Connection with Manufacturing Entrustment, etc., and the payment due date is confirmed in Article 3. For the covered business operators and transactions, distinguish between orders placed on or after January 1, 2026 and transactions to which the former Subcontract Act applies. For covered transactions, the due date must be set within as short a period as possible within 60 days from the date of receipt of the goods (or, for services, the date they were provided), regardless of whether there is an inspection. Being within 60 days does not mean that it is acceptable to pay after a payment date already agreed upon.

Choose where to check according to the type of information. For statutory provisions, check the current text and the history of past versions in e-Gov Laws and Regulations Search; for court decisions, check the full text of the judgment in the courts' case information or a reliable case law database. Even if a case search returns no hits, there may be court decisions not posted in public databases, but you should avoid citing an unconfirmed judgment as legal authority. In addition, ministry guidelines differ in nature from statutes, which are legally binding, so go to the latest published materials to understand their legal character and scope of application.

The order to follow here is to keep AI output as "a foothold for searching" and to confirm the final grounds for judgment in the text of the statute or the actual judgment. Once you open the original text, it becomes easier to notice not only errors in numbers but also missing exceptions and premises.

Prompts for Picking Out What Needs to Be Checked

By using a prompt that has the AI honestly flag uncertain statements from the outset, you can make the subsequent checking work go more smoothly. The following is a prompt that makes the AI flag uncertain citations. It is not a prompt that will always prevent errors, so use it in combination with checking the original sources.

Please explain the following topic, subject to these three conditions.

- If you cite article numbers or court decisions and they are uncertain, clearly mark them "needs verification"
- Do not invent article numbers or case names by guesswork
- If you are not certain, give "only the name of the provision" and do not state the number definitively [Enter your question here]

Having the AI clearly mark items "needs verification" lets you immediately see which parts to focus on when verifying. However, even with this instruction, not every uncertain statement will necessarily be marked, so it is important to treat it as a device that assists with, rather than replaces, the effort of checking.

The next prompt has the AI extract the parts of an answer that require source confirmation. Once these are listed, they serve as a guide when the person in charge opens the sources to check.

From the following text, list only the parts that require confirmation of their sources, such as article numbers, court decisions, figures in laws and names of guidelines. For each item, show both the content being cited and the primary source that should be used to confirm it (e.g., e-Gov Laws and Regulations Search, the courts' website, etc.).

Note that the actual confirmation will be done by a person going to the primary sources. [Paste the text you want to check here]

Using this prompt, the items to check and where to check them are organized out of a long answer, and the work of comparing against the original sources can proceed efficiently. Even when you have the AI itself recheck or review its output, the results of that self-check cannot be regarded as independent evidence. The extracted items must always be confirmed by the person in charge accessing the primary sources.

Common failures in practice include copying article numbers presented by AI into internal documents or replies to counterparties without questioning them, and trusting only the conclusion of a court decision while overlooking differences in the facts. Even if a case name actually exists, if the underlying facts or the procedural issues differ, it cannot be applied as is to your company's matter. It is also important to take care not to carelessly enter confidential company information or personal information into prompts during the checking process. The fact that "the AI answered that way" is not a basis for explanations inside or outside the company, and I think sound practice is to keep to the premise that people bear responsibility for the final confirmation of sources and for the judgment.