Do Not Use Generative AI Answers As Is: Why a Person Must Always Do the Final Check

Hello, I'm Noriaki Asato, Representative Attorney at LegalAgent.

Generative AI can quickly assist with drafting contracts and with preliminary research. In legal practice too, the situations in which it is useful are steadily increasing, such as having it produce a first draft of an NDA (non-disclosure agreement) in a few minutes or having it summarize the key points of a long service agreement in bullet points. I think it can be used for this kind of groundwork, so that the time saved can be spent checking the original text.

On the other hand, if answers created by AI are reused in work as is, there is a risk of presenting terms to the counterparty that differ from your company's policy. Imagine, for example, a situation in which the setting in an AI-drafted clause that "the confidentiality obligation shall survive for five years after termination of the contract" is presented to the counterparty without being checked against internal standards, and it later turns out to conflict with your company's usual standard of three years. The appropriate confidentiality period varies depending on whether your company is the disclosing or receiving party, the nature of the information, bargaining power and other factors, and it is not uniformly fixed by law. However, because AI-generated text is polished, discrepancies with your company's policy are easy to overlook on a simple read-through. Internal staff or the attorneys you have engaged go back to your company's policy and the supporting materials to check.

Errors and Missing Assumptions That Remain Even in Polished Text

A major characteristic of generative AI is that it answers confidently in polished prose even when the content is not factually accurate. This is generally called "hallucination" (plausible-sounding errors). It may seamlessly weave nonexistent article numbers, fictitious court decisions and baseless figures into a plausible context.

In legal practice, accuracy is indispensable. Even if, when asked to review a damages clause, the AI writes that "the party is liable for nonperformance under Article 415 of the Civil Code," you need to go to the official primary source to confirm whether the article number is correct and whether that provision can be applied as is to the case at hand. AI can take a certain amount of context into account when combined with tools that search internal rules and reference materials, but even models with the ability to search external materials can make typos and mix things up. It is important to treat AI answers strictly as "drafts that require verification" and not to place unconditional trust in them.

Further, AI is not a mechanism that can grasp the background of each matter or your company's particular circumstances on its own. Information such as the history of past negotiations, the balance of power with the business partner and management's risk tolerance must be provided through prompts, reference materials you have authorized it to use and the like. Even after the information is provided, whether it has been correctly reflected must be checked separately.

For example, if, in a service agreement, the counterparty proposes that "the cap on damages shall be one month's service fees," and your company is in the position of seeking damages from the counterparty, the AI might point out that "the cap is low, so there is room for negotiation." However, while there may be a risk of substantial actual damages from information leaks or intellectual property infringement even if the service fees themselves are small, if the transaction is a strategic matter for which an early start should be prioritized for your company's new business, you may choose to agree even to somewhat unfavorable terms.

Similarly, even if the AI advises regarding a termination-without-notice clause that "it is desirable to provide a cure period to avoid one-sided disadvantage," the realistic risk may be assessed as low in light of the relationship of trust with the counterparty and the realities of the transaction. However, you cannot necessarily rest easy by accepting termination without notice unconditionally just because there is a long-term business relationship, and the decision should be made carefully, taking into account how specific the termination grounds are, the cure procedures and the impact of termination of the contract. Evaluating contract terms also requires considering what could happen in the actual transaction. Whether to adopt a proposal must be decided by a person who understands the circumstances.

Items That Deserve Especially Thorough Checking

When checking AI output, rather than scrutinizing every statement uniformly, a realistic approach is to vary the level of checking according to the nature of the clause and the size of its impact. The check items that deserve particular attention in practice are as follows.

• Checking facts (article numbers, court decisions, figures, dates, proper nouns) against primary sources
• Comparing general principles with your company's business situation and negotiation policy
• Confirming the appropriateness of the wording and content as language to be presented to the counterparty
• Checking clauses directly tied to your company's disadvantage, such as damages, termination and ownership of intellectual property rights
• Detailed review in matters involving large transaction amounts or handling personal information or trade secrets

For items such as the survival period of confidentiality, the cap on damages and who owns intellectual property, a single difference in a number or subject can greatly change your company's legal position. For such critical clauses, a careful attitude is essential: do not reuse the AI's output as is, but check it word for word against the primary sources and your company's standard criteria.

In addition, the fact that "the AI pointed this out" must not be used as the basis for a decision in internal explanations or in negotiations with the counterparty. The mere existence of an AI answer is not grounds for the facts or legal interpretations written in it being correct. In accordance with the company's approval authority, the decision on whether to adopt something is made after checking the primary sources and the circumstances of the transaction. If a dispute arises from an incorrect decision, responsibility is judged individually in light of the job authority and specific involvement of the people concerned and the provisions of the transaction agreement.

Requesting Groundwork and Designating Who Checks

When asking generative AI to do groundwork, also decide who will check it afterward. This is to make clear who reads the original text and who approves the terms.

AI's strengths come into play in tasks such as summarizing long contracts, identifying issues to consider, drafting first versions of revised clauses and preparing first drafts of internal explanatory memos. By contrast, the areas that people should handle are setting policy in light of your company's risk tolerance, deciding whether to adopt counterparty-facing comments prepared by the AI, assessing issues on which legal interpretations differ, and making the final decision to agree.

Taking the review of a non-disclosure agreement (NDA) as an example, you have the AI do the groundwork by instructing it to "list points regarding the scope of confidential information, restrictions on use for other purposes and the survival period that could be disadvantageous to the receiving party." The person in charge reviews the output, selects which points to raise in line with your company's position and the realities of the transaction, and then finalizes them as comments to the counterparty. In this approach, the person in charge gets help with the draft while keeping a record of the grounds checked and the reasons for adopting each point.

Prompts That Leave Uncertain Points Flagged

To reduce the effort of checking, it helps to build in, from the stage of the first instruction, the premise that the AI must flag uncertain statements. First, as a basic form, here is an example prompt that has the AI self-report points it is not sure about.

As an assistant supporting corporate legal work, please list the possible issues in the following contract clause. There are three conditions:

- When citing article numbers or court decisions, clearly mark uncertain ones as "needs verification"
- Do not make up numbers or facts based on guesses
- On the assumption that a person will make the final decision, present points as "issues" without stating conclusions definitively[Paste the clause here]

Specifying things this way makes it easier for the AI to point out vague grounds itself, and makes it easier to prevent the person in charge from overlooking points that need to be verified. However, giving this instruction does not guarantee that every uncertain item will be flagged, so it does not mean that checking important points can be skipped.

Next is a prompt that has the AI review a draft comment to be sent to the counterparty once more. Even after the automated review, a check by a person remains.

The following text is a draft of contract revision comments we plan to send to the counterparty. As a pre-send check, please list any concerns from the following perspectives. Note that a person will make the final decision on whether to send it and adjust the wording.

- Whether our confidential information or internal circumstances have been written carelessly
- Whether the wording is so strong that it risks hardening the negotiation
- Which quotations of facts or provisions need to be verified[Paste the draft comment here]

For points identified in this review as well, do not treat the AI's answer as absolute; a person must always go to internal materials and public information to confirm the facts. It is also important to check in advance, before entering a prompt, that it does not contain your company's confidential information or personal information.

The closer the deadline, the more tempting it is to skip the checking procedure, but do not commit to unverified terms with the counterparty because the deadline is near. Even when pursuing efficiency, I think that safe use in practice comes not from cutting the checking procedure itself, but from designing instructions to the AI so that it produces output in a form that is easy to check.