Designing Log Audits and the Allocation of Responsibility for AI Agents

Hello, I'm Noriaki Asato, Representative Attorney at LegalAgent.

When an AI agent processes something incorrectly, can you explain after the fact what the system referred to, which tools it called, and on whose judgment the action was executed? If you introduce an AI agent into your operations without being able to answer this question, it becomes difficult to investigate the cause and to explain the matter internally and externally when trouble occurs. In the legal review for introducing AI agents, you keep records that can explain how things happened and decide the roles of staff and vendors in advance.

Behind this is a change in how AI is used in business. AI drafts emails and customer responses and searches internal databases. It also calls external tools to open tickets and prepares proposed revisions to contracts, and the range of tasks handled by AI is expanding. As a result, there are more situations in which processing proceeds where human eyes do not directly reach.

That said, it is not simply a matter of keeping unlimited logs of every operation. Logs may contain personal information, trade secrets, and employment-related information, and the audit logs themselves may create risks of leakage or excessive monitoring of employees. It is important to balance the purpose of use under the Act on the Protection of Personal Information (APPI), the duty to endeavor to erase personal data that is no longer needed, and the retention requirements for audits and dispute response, and also to keep track of developments in terms of service and laws and regulations.

Defining Log Audits and the Allocation of Responsibility

An AI agent is a system that does not merely answer questions but autonomously searches for information, links external tools, and carries out multi-step work. A typical example is a sequence of processing in which the system reads a contract, checks it against internal templates, prepares a proposed revision, and notifies the person in charge.

A log audit means recording the operating history of an AI agent so that objective verification can be performed after the fact. The records cover user operations, referenced data, tool executions, whether approval was given, errors that occurred, and similar items.

The allocation of responsibility means clarifying the scope of work handled by AI and then separating the roles and legal responsibilities borne respectively by the deploying company, the actual users, and the vendor providing the tool. It does not mean attributing legal responsibility to the AI itself. Where responsibility should be examined changes depending on whether the AI merely presented a proposal, a human checked and approved the content, the action was executed automatically, the cause was a model change or defect on the vendor's side, or the cause was a user's input error.

With a chat AI that only answers questions, the flow of processing is relatively simple, but where responsibility lies differs depending on how it is used. AI agents, in contrast, work directly with other systems to carry out tasks, so they are by nature prone to blurring where responsibility lies. That is precisely why design before deployment is essential.

Isolating Causes and the Difficulty of After-the-Fact Verification

The reason log audits and the allocation of responsibility should be designed in advance is that an AI agent's mistakes are difficult to see from the surface.

Consider, for example, a case where an AI agent has prepared a proposed contract review. When there is an erroneous interpretation of a provision, was the AI's output itself wrong, was the past template it referred to outdated, was the instruction prompt inadequate, did the person in charge neglect to check, or did a model update by the vendor have an effect? To isolate the cause after the fact, the records at each stage provide the clues. Without logs, identifying the cause and considering measures to prevent recurrence become difficult.

The same applies to customer service. If the AI prepares a draft response to an inquiry, the person in charge sends it after only a cursory check, and an error is later discovered, you will need to verify where the problem lay: in the content generated by the AI, the freshness of the FAQ it referred to, or the scope of the staff member's check.

Managing access rights is also important. Situations can occur in which the AI searches for information that the staff member personally has no right to view and includes it in a response, or mistakenly refers to data from outside the department or information about another company's matter. If you can check by combining logs and access controls, you will be able to grasp the cause and take countermeasures should such a situation arise.

If operations begin with the allocation of responsibility left unclear, attitudes toward use in the field tend to become polarized. Some people will trust the AI blindly and skip checks, while others will avoid using it for fear of where responsibility lies. Deciding who checks proposals, how far automatic execution is permitted, the approval flow for high-risk tasks, and who is responsible for external explanations in the event of an incident makes it easier for people in the field to judge the scope within which they can use it.

Specific Items to Check in Log Design

The first step in design is to classify the business tasks entrusted to the AI agent. Check whether it will be limited to writing, summarizing, and internal search. If it will be entrusted with updating tickets, revising contracts, or preparing customer responses, also decide whether it will go as far as sending externally on its own. The nature of the tasks determines the granularity of logs and the scope of permissions required.

The scope of information that can be referenced is set based on the principle of least privilege and the scope of the business. When giving access to internal knowledge, contracts, CRM, or accounting data, the focus is on how to map the user's permissions to the AI's permissions. If the AI is given broader permissions than the user, information that the user should not be able to see can be extracted via the AI, so appropriate separation of permissions must be achieved for each task.

Decide the log items needed for audits and incident response. The basics include the date and time of use, the user, the instructions entered, the data referenced, the external tools called, the output generated, the content of human modifications, the record of the approver, and whether there was any error information.

Retention periods and viewing permissions are set in stages according to the content of the logs. While some logs must be retained for a certain period for audits and dispute response, data containing personal information or trade secrets also calls for consideration of prompt erasure once no longer needed, to prevent use for other purposes. Viewing permissions should also be limited according to whether the purpose is auditing, incident response, or quality improvement.

The granularity of what is recorded also needs consideration. Storing the full text of contracts, customers' personal information, or trade secrets contained in prompts as they are could turn the log storage location itself into a new leakage route. Whether to keep the original text as is or to apply masking or summarization is decided by weighing the need for auditing against information management risk.

It is realistic to expand the scope of permitted automatic execution cautiously. At the initial stage, limit it to drafting, and expand permissions in stages to updating internal tickets and then to limited external sending. When external sending is executed automatically, it is often difficult to roll back after sending, so it is safer to refrain from granting broad permissions from the outset. Prepare an emergency stop mechanism, steps awaiting approval, and procedures for sending back in advance.

Furthermore, unless you also decide on auditors who regularly review the logs, a reporting route when anomalies are detected, and a system for feedback to business departments, you will end up merely accumulating records.

Where external transmission to external APIs, cloud storage, or foundation model providers is involved, confirm that you can reliably record the destination, the content sent, and whether approval was given. Also decide in advance who will check which logs and judge whether reporting to customers or regulators is necessary when an incident occurs in which misdirected transmission or reference beyond permissions is suspected.

How Roles Differ Among Internal Departments and Vendors

For the deploying company as a whole, the central point is clarifying who the final decision-maker will be. Who checks a contract revision prepared by the AI before it is sent to the counterparty, and to what standard is a draft customer response checked before it is sent? For tasks involving important judgments, a design that makes human checking and approval mandatory is essential.

Business departments should understand that, while they use AI as a means of improving efficiency, accountability to outside parties and responsibility for decisions will be examined in light of specific duties and operations even when AI is used. A company or staff member is not relieved of responsibility simply because "we sent it as the AI proposed." The judgment is made in light of the specific duties and the state of checking.

Legal and compliance departments play the role of regularly reviewing the scope of application of AI agents. An AI initially introduced only for writing may expand, as the business develops, into external sending, contract revision, and customer service. Before expanding the scope of use, review the log items, approval rules, and handling of personal information.

For information systems and security departments, the central issues are separation of permissions and secure management of logs. If an AI agent is granted excessive permissions equivalent to an administrator's, the impact of a malfunction will spread across the entire system. Consider setting least privilege and providing an emergency stop button.

In the relationship with AI vendors, the issue is to clarify in the contract the limits of the service provided and the users' duty to check. Set out in the contract to what extent the vendor bears responsibility when erroneous output or system failures occur, the degree of checking required of users, and the scope of logs that can be disclosed.

From a labor management perspective, how far employees' operation logs are subject to audit becomes an issue. It is likely desirable to take care over the boundary between the legitimate purpose of business auditing and excessive employee monitoring, and to make the purpose and scope of log collection clear to employees through internal rules or similar.

The Limits of Reviewing Rules with Generative AI

Using generative AI to review drafts of terms of service, security materials, and internal rules is effective in itself. It can efficiently identify omissions in provisions on log retention periods, disclaimers, and audit rights.

However, simply having AI read the text of the rules does not reveal operational risks that reflect reality. Which internal systems does the AI agent connect to, what permissions does it have, which data does it refer to, and how far does it carry out automatic processing? Unless such realities of operation in the field are reflected, the review results will remain general advice. Even if there is no problem in the rules, the risk is not resolved if the access permissions on the system side do not match the rules.

You also need to bear in mind that the AI's output does not always reflect the latest laws, regulations, and guidelines. The APPI, AI-related rules, and the terms of service of external services are revised from time to time. In examining log audits and the allocation of responsibility, it is important to proceed by comparing the latest information at the time of deployment with the usage environment in the field.

When LegalAgent receives consultations about AI agents, we focus not only on confirming the tool's functions but on whether responsibility and the course of events can be explained when trouble occurs. Who checks what, which records remain, and by what procedure can processing be stopped? I believe that if AI is to be built into business workflows, this explainability should be built in from the initial design stage.

Frequently asked questions

What are log audits and the allocation of responsibility for AI agents?

A log audit means recording the history of operations so that they can be verified after the fact, and the allocation of responsibility means separating the roles and legal responsibilities of the company, users and vendors. Because AI agents autonomously search for information and work with external tools, where a mistake occurs, it tends to be unclear where responsibility lies. I therefore think it is necessary, before deployment, to design a system that keeps records and allocates responsibility.

What items and risks should be considered in designing logs for AI agents?

Set items such as records of the date and time of use, instructions, data referenced, tools executed, output, and corrections and approvals, while taking leakage risks into account. This is because storing full contract texts, customers' personal information, or trade secrets contained in prompts as they are could turn the storage location itself into a new leakage route. Weigh the need for auditing against information management risks and decide on masking, summarization and restrictions on viewing rights.

If an AI agent performs incorrect processing, is the company relieved of responsibility?

The company's legal responsibility is not waived merely because it acted as the AI suggested. This is because legal responsibility does not attach to the AI itself, and accountability to external parties and responsibility for decision-making are judged in light of the specific obligations and actual operations. I think that, for work involving important decisions, human review and approval should be mandatory, and it should be made clear who reviewed and approved the proposal.