Logging an AI System So an Incident Can Be Reconstructed
When an assistant does something wrong, you need to know what it saw, what it was asked and what it called. Logs are the only way to answer.
On this page
Last month an assistant gave a customer the wrong refund policy, and the team that owns it could not say why. The model, the instructions, the retrieved articles and the tool outputs were all in play, and none of them had been recorded in a form that could be read back. The incident then became a guessing exercise, which is the outcome good logging exists to prevent.
What standard logs miss
Ordinary request logs record the time, the endpoint, the status code and perhaps the user. That tells you a request happened and how it ended. It does not tell you what the model saw, what it was asked to do, which documents it read or which tools it called. An AI incident cannot be reconstructed from that alone.
The useful additions are the input as the model received it, the version of the instructions used, the retrieved documents with their identifiers, the model and its settings, every tool call with its arguments and result, the final output, and any filter decisions made on either side. Without these, an investigation stalls at its first question. Did the model misread the policy, did retrieval return an outdated article, or did a template version send the wrong instructions? Each answer points to a different fix.
Record the versions
An AI response is produced by a combination of parts: a model, a prompt template, an instruction set, a retrieval index snapshot, a tool schema and a safety configuration. Log the version of each, and log the combination for every request. Without that record, a change in behavior cannot be matched to the change that caused it. Store a content hash of each template and instruction set rather than only a name, since a name can be reused after an edit while the hash cannot.
Model identifiers need care. Some providers offer a general alias that can point to different underlying models over time. Record the identifier the provider returns where one is available, and note in the log whether an alias was used. Even with identical settings, output can vary between calls, so the log supports reconstruction but does not promise that a fresh run will produce the same words.
Privacy, retention and access
Logs of AI systems contain personal and sensitive data, often more than developers expect, because users type whatever they think matters. Decide the purpose of each log before writing it, and collect only what that purpose needs. Redact at write time where you can, particularly for credentials and identifiers, which should never reach a log in clear form.
Set retention by purpose rather than by habit. A debugging copy of a transcript may need a short life, while an audit record of tool actions may need to last longer. Keep full transcripts in a more restricted store than summary records, and log every access to that store. Whether keeping personal data in logs is lawful for your purposes is a question for your privacy and legal teams, and it should be answered before the logging is switched on.
Design the record for reconstruction
Give every request an identifier and carry it through every step: retrieval, the model call, each tool call and post-processing. Record timestamps on one clock, so the order of events can be trusted. Pseudonymize user and session identifiers where an investigation does not need the real ones, and keep the mapping in a separate system.
Write the log even when a request fails, because failures and timeouts often explain the strangest behavior. Store logs in a form that resists alteration, such as append-only storage with access controls and a record of who read them. If you sample logs to control cost, define in advance which requests are always kept in full, such as those that triggered a filter, touched a write tool or came from a high-impact feature. Sampling that drops those cases leaves gaps exactly where an investigation needs them.
Use the logs during and after an incident
During an incident, the logs should answer a short list of questions quickly: what did the model see, what did it do, which users were affected, which versions were live, and when did the behavior change. If the logs cannot answer those in order, the schema is incomplete. The goal is a team that can build a timeline without first asking developers to reproduce the problem.
The same records support detection. Alerts on unusual tool sequences, on bursts of reads across unrelated records and on outputs that match patterns you have flagged can catch a problem before a customer reports it. Feed the findings from each incident back into the schema, so the next investigation has the data the last one lacked.
A tabletop exercise
Take one assistant feature and write a logging schema for it, with a field list. For each field, note its purpose and its privacy class, such as public, internal or restricted. Then run a tabletop exercise. Give a colleague a report of a wrong answer, with only the logs in front of them, and ask them to answer three questions: what the model was given, what it did and which version produced the answer.
Record every place they cannot answer. Each gap is a field to add or a link to repair. The exercise is worth more than a schema review alone, because it tests whether the logs answer the questions an incident will actually ask.
