PII in LLM Outputs: Why Input Filtering Is Not Enough

Most teams start AI governance at the input: scan the prompt, mask what is sensitive, forward the rest. That closes the obvious hole and leaves a subtler one open — the model itself can produce personal data in its answer.

Where output PII comes from

• Retrieval: a RAG pipeline pulls a customer record into context and the model quotes it back
• Tool calls: an agent queries an internal API and pastes the raw result into the response
• Memorization: fine-tuned or long-context models reproduce training or session data
• Reconstruction: the model infers and restates a masked value from surrounding context

None of these are visible to an input-only filter. If the answer is then shown in a support console, emailed, or handed to another agent, the exposure is identical to sending the data to the provider in the first place.

Two modes: shadow and enforce

Turn output scanning on in shadow mode first: detections are logged, nothing is altered, and you get a real measurement of how often your pipelines emit personal data. Once the false-positive rate is acceptable, switch to enforce, where the configured action — mask, tokenize or block — is applied before the response reaches the user.

The streaming limitation, stated honestly

With server-sent events the provider emits tokens one by one and each token is already on its way to the client. A partial identifier can cross the wire before it is recognisable as one. Any vendor claiming full masking over SSE is describing buffering, which removes the reason to stream at all.

The workable design is: SSE is audit-only — detections are recorded as incidents so a DPO can review them — and any flow that must guarantee masking uses a non-streaming call. In Privaro that means the relay endpoint for streaming and POST /v1/proxy/protect-output when the response must be sanitized before delivery.

What to log

An output incident is only useful if it records direction, entity types, the policy that matched, the action taken and whether the content actually reached the user. That last field is what separates "protected" from "leaked" in an audit, and it is the number your DPO will ask for.

Next step

See how masking works on the request side in how to mask PII before sending prompts to OpenAI, or read the PII detection API reference.