Secrets, Keys and Personal Data in Prompts

Most people paste a stack trace into a chat window without checking what else it contains. A configuration fragment, a customer name in a log line or a connection string in an environment dump can travel with the part that actually mattered. Once the prompt is sent, the data has left the place where you could control it.

What counts as sensitive here

Sensitive material in a prompt falls into four groups. Credentials come first: API keys, access tokens, passwords, and connection strings with embedded passwords. Personal data follows, including names, contact details, account identifiers, and health or financial information. Confidential business material includes unreleased plans, customer lists, contract terms and internal financial figures. The fourth group is system detail: hostnames, internal addresses and architecture descriptions that help someone plan an attack, even when they look harmless.

Derived material counts too. A summary of a contract, a set of error counts or a list of internal project names can reveal more than any single value, even when no individual item is a secret. Ask what someone could learn from the whole prompt, not only from its obvious parts.

Whether a given item is regulated depends on the jurisdiction, the contracts in place and the people the data describes. The list is a starting point for your own classification, not a legal answer.

Follow the data after it leaves

A prompt rarely stops at the model. It may be written to your application logs, captured by tracing or monitoring tools, stored by the model provider under a retention policy, indexed if it feeds a retrieval store, or copied into a support ticket when someone reports a bad answer. Each of those is a separate location, with its own access controls and its own retention.

Check each provider's terms for retention, for any use of data in training, and for the regions where processing happens. These settings differ between services and change over time. Verify them for the specific service and plan you use rather than relying on memory or a secondhand summary, and record what you found with a date.

Redact before the request leaves

Redaction removes or replaces sensitive values before a prompt is sent. Pattern matching works well for values with a known shape, such as common key formats or card-like numbers. Named-entity detection can find personal names and places, with more errors. Allow-lists are often more dependable than block-lists. Send only the fields a feature needs and drop everything else at the application boundary.

Replace values with stable placeholders rather than deleting them. A placeholder such as a generic customer label lets the model reason about the relationship between items without seeing the real identifier. Keep the mapping inside your own system, so the placeholder can be resolved after the response returns.

Redaction is probabilistic, and its failures are quiet. Test it against a sample of realistic data, including the messy cases such as keys pasted inside code blocks or identifiers split across lines. Count what slips through, and fix the rules for each category that leaks.

Keep secrets out of prompts altogether

The best redaction is never needing to redact. API keys should live in a secret store and be used by server-side code. The model does not need a key's value to call a tool. The tool's backend holds the credential and makes the call, and the model receives only the result.

Assume that anything in the model's context can be repeated back to a determined user. A system prompt is therefore not a vault. Keep secrets out of it, and do not rely on an instruction telling the model to keep them hidden. If a key has ever appeared in a prompt, a log or a trace, treat it as exposed and rotate it. Rotation is usually cheaper than an investigation that tries to prove nobody saw the value.

Review what has already been sent

Build a routine review of what the system already stores. Search logs and trace stores for common key patterns and for personal identifiers that should have been redacted. Check retention settings against what the business has decided is necessary. Confirm that access to transcripts is limited to people who need them, and that the access itself is logged.

Repeat the review after changes, such as a new feature that adds a data source or a new logging tool. Each new component is a new place where prompts can end up.

Look beyond the application as well. Shell history, notebook outputs and copied error messages in a team's chat channel all carry prompt contents to places the application never sees. Developers often paste production values into a debugging session because it is the fastest way to reproduce a problem. Give them a sanctioned source of realistic test data, so the habit is replaced rather than merely forbidden. A rule that works against a habit people have a reason for is more likely to hold.

A redaction exercise

Create fifty synthetic prompts that resemble your real traffic, with invented names, invented keys in the right format and invented identifiers. Run them through your redaction step and count what leaks. For each leaked category, write the rule that would catch it, then run the set again to confirm the rule works.

The exercise also shows the cost of the approach. Aggressive redaction can strip context the model needs, so note any prompt whose answer became worse after redaction. The right setting protects the data and keeps the feature useful, and you find it only by measuring both.