Document security / What is inside it
Why a black rectangle over text does not redact it
The most consequential misunderstanding in document handling: covering text with a shape hides it visually and leaves it entirely intact in the file.
The exposure
A PDF stores text and graphics separately. Drawing a filled rectangle over a run of text adds a graphic on top; it does not alter the text underneath, which remains in the content stream and comes out of any extraction, any copy and paste, and any search. The same is true of white text, of a covering image and of CSS that hides an element after the fact. This has produced real disclosures in published documents repeatedly, and it keeps happening because the visual result is completely convincing.
The decisions
Reasons rather than a description of the code. Each one has a default that is fine for a page and wrong for a file somebody keeps.
- Redact by not rendering. The only reliable approach in a generated document is to omit the content before it reaches the renderer, so it was never in the file.
- Never rely on a covering shape, white text, opacity or a clipping region. Every one of those leaves the text extractable.
- Never rely on CSS visibility for a secret. Hidden elements can still be in the document, and are trivially recoverable.
- Test extraction rather than looking at the page. Extract the text of the finished file and assert the redacted value is absent, which is the only check that means anything.
- Where you must redact an existing document rather than regenerate it, use a tool that removes the underlying content, and verify by extracting afterwards.
- Remember it applies to images too. Covering a face in a photograph does not remove the pixels underneath unless the image itself was altered before embedding.
In practice
A fragment, with the thing that goes wrong kept in a comment where it is the more instructive half.
// The only reliable redaction in a generated document: never render it.
function redactedInvoice(invoice, viewer) {
return {
...invoice,
// Omitted, not hidden. It is not in the markup, so it is not in the file.
bankDetails: viewer.canSeeBankDetails ? invoice.bankDetails : undefined,
};
}
/* None of these redact anything. All of them look completely correct
on screen and leave the text in the file, extractable by copy and
paste, by search and by any extraction tool:
.secret { background: black; color: black; } still there
.secret { color: white; } still there
.secret { opacity: 0; } still there
.secret { visibility: hidden; } still there
.secret { display: none; } usually absent, but
do not rely on it
<rect fill="black" ...> drawn over the text still there
The same applies to images: covering a face with a shape leaves the
pixels underneath. The image itself has to be altered before it is
embedded. */
// The only check that means anything.
test("redacted values are absent from the file", async () => {
const pdf = await render(redactedInvoice(invoice, viewerWithoutAccess));
const text = await extractText(pdf);
expect(text).not.toContain(invoice.bankDetails.iban);
});What people do instead
Redacting in CSS because the document already exists and regenerating it is inconvenient. The result is visually perfect and discloses everything to anyone who selects the text, which is how this appears in the news every year or two.
How this is found out
By anybody who copies text out of the document, which is a two-second action requiring no skill. That is why this failure becomes public rather than staying internal.
Frequently asked
Does this page tell me what the law requires?
No, and deliberately not. Retention periods, erasure obligations and residency rules vary by jurisdiction, by industry and by the kind of document, and they change. What these pages describe is the shape of the problem and the mechanisms a system needs in order to implement whatever answer your own advisers give you. Where a genuine tension exists, such as an immutable record against a right to erasure, it is named as a tension rather than resolved.
Why is so much of this about the contents rather than access control?
Because access control decides who can obtain a copy and has no opinion at all about what happens to the copy. Once a document is on somebody's laptop, forwarded to a colleague or printed, every control listed here has already stopped applying to it. What is inside the file is therefore the part that keeps mattering, which is the opposite of the balance you would strike for a page.
How much of this applies at a small volume?
Most of it, because these are decisions rather than infrastructure. Redacting by omitting rather than covering costs nothing. Deciding what goes on a template costs one review. Classification is one object in code. Legal hold and an audit trail are the two that take real work, and both are far cheaper to build before they are requested than under the deadline that comes with the request.
Related security topics
The decisions that depend on each other, then the rest of the same group.
Personal data in a document that will be emailed
What a document discloses once it is attached to a message, which is more than the recipient asked for and travels further than the message.
What metadata a generated PDF carries, and how to check
The fields a document format carries alongside its visible content, which are invisible on the page and present in the file.
Internal information that ends up printed on a customer document
The values that reach a customer-facing document because they were available, which are usually internal identifiers, judgements and debugging aids.
Files embedded inside a PDF, and what they can carry
A PDF can contain other files inside it, which is useful for machine-readable payloads and is a way for content to travel without being visible.
Classifying generated documents so handling rules can apply
Attaching a sensitivity level to each kind of document, which is what makes every other rule in this cluster applicable rather than aspirational.
Every document security topic
The full list, grouped by access, contents and what happens afterwards.
What this API actually does
The options and endpoints these decisions are built on, one page each.
Most of these are decisions rather than features, and the cheapest time to make them is before the first document is delivered rather than after one reaches the wrong person.