Keeping Secrets Out of Prompts

Here's a scene that plays out constantly. Someone's config file won't load. They copy the whole thing, paste it into an AI chat, and ask what's wrong. The model spots the typo in about two seconds. Great result.

The file also had a database password in it. And an API key. Nobody meant to share those. They were just in the file.

Why people do it

I don't think this is a people problem, or at least not mainly. Pasting the whole thing is the fastest way to get a useful answer. If you trim the file first, you might trim the part that matters. If you swap the key for "XXXX," you have to remember to swap it back. When you're mid-problem and something's broken, convenience wins almost every time.

And it's not just keys. It's a customer's email pasted in with a request to "help me write a polite reply." A spreadsheet of names and phone numbers someone wants summarized. A log file full of IP addresses. A patient note someone wants reworded. Every one of those is a reasonable thing to ask an AI for help with, and every one carries data that shouldn't go wandering.

Apps built on top of models make this bigger, not smaller. Now it's not just a person pasting. It's your app assembling prompts automatically from records, tickets and documents, often with no human looking at what's in them.

Why "before it leaves" matters

When a prompt goes out to a model provider, a copy of it lives somewhere you don't control. Maybe it's logged. Maybe it's kept for abuse monitoring. Maybe it's handled perfectly. The trouble is you can't un-send it, and "we'll clean it up afterward" isn't a plan when the thing that needs cleaning up is on someone else's servers.

So the useful moment to catch sensitive data is before the request goes out the door, not after. That's where G8KEPR does it: the PII and secret filtering lives in the AI Gateway, inside the request path, and it redacts sensitive values before the prompt is sent on to the provider. The model still gets enough to do its job. It just doesn't get the actual password.

What it looks for

Out of the box it covers the kinds of things you'd expect: email addresses, IP addresses, Social Security numbers, card numbers, medical record numbers and so on. On top of that, G8KEPR supports custom recognizers, which is the part that matters for real organizations. Every company has its own sensitive formats. Internal account numbers, employee IDs, project codes. A generic list will never know about those, so there has to be a way to teach it.

A quick definition, since it comes up: PII is personally identifiable information, anything that can point back to a specific person. Secrets are the machine kind: passwords, API keys, tokens and anything else that grants access to something.

Why it's in the gateway

I could have built this as a plugin each app has to remember to install. I didn't, for the same reason I put most of the checks in one place.

If every team building an AI feature has to wire up their own redaction, some will do it well, some will do it halfway, and one will skip it because the deadline was Tuesday. Putting the filter in the gateway means every request that goes through it gets the same treatment, regardless of which app sent it or which model it's headed to. I make the broader case for that in One Gateway, Many Models.

It also runs inside your own VPC, which matters here more than anywhere. A tool that protects your sensitive data by shipping it to a third party for inspection would be missing the point. The reasoning is in Why It Runs in Your VPC.

What it isn't

A filter like this is a seatbelt. It's there for the moment someone does the natural, convenient, slightly risky thing. It's not a substitute for deciding, as an organization, what's okay to send to an AI service in the first place, or for telling people about it.

But people are going to paste. I'd rather catch it quietly than pretend they won't.