Engineering

The model proposes, the code disposes

By Barton Hall Software · 21 September 2026 · about 8 minutes

Never let a generative model rewrite the document

The obvious build is to hand the document to a language model and ask for a redacted version back. We tried it early. It produces something that looks right and is quietly wrong: numbers drift, table rows get dropped, a paragraph is paraphrased, and a date that was never sensitive comes back changed. You can't diff your way out of it, because the model rewrote the parts it was supposed to leave alone.

So the model's only job is to propose spans: this range of characters is a person's name, that one is an account number. Everything after that is ordinary deterministic code operating on the original bytes. The document that comes out is the document that went in, minus exactly the spans that were removed.

The nice property of this split is that the failure modes separate. A detection miss is a leak, which we measure. A disposal bug corrupts a file, which tests catch. Mixing the two makes both invisible.

What the deterministic half has to get right

"Delete the text" sounds like one line of code. In real documents it isn't.

• PDFs hold text as positioned glyphs. Removing a name means removing the glyph runs and then repainting the region, not drawing a black rectangle over them. If you only draw the rectangle, the text is still in the file and anyone can select it.
• Scans have no text at all. The name is pixels. It has to be found by text recognition and then physically erased from the image.
• Word files hide text outside the body. Hyperlink targets, screen tips, field instructions, document properties, comments, and tracked changes all carry names the model never sees as prose, so they are scrubbed separately and then fed back through the same checks.
• The same value is spelled several ways. A name in a URL is percent-escaped, a phone number has different spacing, an address is split across lines. The matcher normalises through case, punctuation, and percent escapes, so one identity is one value however the document spells it.

There's also a rule we hold to when something goes wrong: degrade to over-hiding, never to leaking, and never to refusing. If a page can't be read reliably, the page is hidden whole rather than passed through. An over-redacted page is annoying. A leaked one is the reason you used the tool.

The numbers

The metric is leak rate: of the personal identifiers present in a document, what fraction survives into the redacted output. Lower is better. We score the final output, not the model's raw proposals, because that's what a user actually receives.

* measured on the previous shipped version of our model; the current one hasn't been re-run on TAB. Single runs, roughly ±1.6 points of noise. These are our own measurements, not a third party's.

We also keep a small Australian gate document with 21 distinct identifiers in it, including the identifier formats that general-purpose models tend not to know: ABNs, ACNs, and local reference numbers. Our model catches 21 of 21 with no leaks. openai/privacy-filter catches 5 of 21.

The honest caveats. That gate is one document, which is a small sample, and our training data is Australian-flavoured, so it's a home fixture. The mirror image is also true: openai/privacy-filter scores far better on AI4P, which it was trained on. Benchmarks reward whoever practised on them. The reason we keep all four is that no single one is trustworthy alone.

Precision matters as much as recall

A redactor that hides everything has a perfect leak rate and is useless. So the same gate checks the opposite direction: every financial figure in the document must survive, and every public organisation name must be kept. A statement where the amounts have been blacked out can't be analysed by anyone, human or model.

Deciding which names to keep turned out to need its own machinery. Detecting that something is an organisation is a regex problem, because legal forms are spelled consistently. Deciding whether an organisation is public knowledge or private to you is a lookup, with an important default: unknown organisations get redacted. A tax office is public. Your employer is not.

The quantisation lesson

This one cost us a week and is the part we'd most want to read in someone else's write-up.

To ship a 1.7B model inside a desktop app you quantise it. We tried 4-bit. The leak rate barely moved on aggregate, which is exactly the trap, because the failure it introduced wasn't statistical. At 4 bits the model started fusing adjacent span boundaries: two identifiers next to each other would come back as one span with a boundary in the wrong place, and the piece outside the boundary survived into the output. A real leak, invisible in the average.

6-bit is the knee for this model. It costs about a gigabyte more on disk and it holds boundaries. If you're quantising a model whose output is character offsets rather than prose, check the boundaries specifically. Aggregate quality metrics will not show you this.

Two engines, because one engine can't check itself

Scanned pages go through Apple's Vision framework, which is fast and gives word boxes. But an OCR engine that produces the text cannot also be the judge of whether it read the page correctly. When it garbles a line, it garbles it confidently.

So a second, independent engine reads the same pixels, and disagreement between them is treated as a signal rather than noise. Characters that can't occur in the source alphabet condemn a text layer outright. Suspicious density is confirmed against the actual pixels before anything is trusted. Where the engines disagree about a region, the region is hidden.

What we don't claim

No automated detector catches everything. Ours doesn't. Any product in this category that tells you otherwise is selling you a number it can't support, which is why the app always shows you what it found and what it kept before you share anything, and why the leak numbers above are published with their caveats instead of rounded into a marketing claim.