Operations / What it costs and what it can take
Degrading document generation instead of failing outright
Producing something useful when the full document cannot be made, which is sometimes right and frequently worse than failing.
Why the default answer is wrong here
Degradation is standard advice and it is dangerous for documents specifically, because a degraded document is still a document. A page that renders without its stylesheet, an invoice missing its logo, or a statement with a placeholder where a chart should be all look like finished artefacts and will be treated as such by whoever receives them. So the question is not whether to degrade but whether the degraded output is safe to hand to a person, and for most business documents it is not.
The decisions
Reasons rather than a description of the code. Each has a default that is defensible in general and wrong for documents specifically.
- Default to failing rather than degrading. A missing document is recoverable and a wrong one that was accepted is not.
- Degrade only where the missing element is genuinely decorative and its absence cannot be mistaken for a statement about the content.
- Never degrade a value. A missing total, date or identifier makes the document wrong rather than plainer, and no rendering trick makes that acceptable.
- Where a document is degraded, mark it visibly, so nobody downstream treats it as complete.
- Prefer deferring to degrading. A document that arrives complete an hour later is almost always better than one that arrives incomplete now.
- If a fallback exists, test it as carefully as the main path, because it runs during an incident when nobody is examining the output.
In practice
A fragment, with the thing that goes wrong kept in a comment where it is the more instructive half.
/* The decision, per element, made in advance rather than at render
time:
brand logo missing degrade. Decorative; absence means
nothing about the content.
chart image missing do not degrade silently. A statement
with a blank where a chart was reads as
a statement about the data.
stylesheet missing fail. The document is unrecognisable
and will still be sent.
any value missing fail. A blank total is a wrong document.
*/
async function renderInvoice(invoice) {
const assets = await loadAssets(); // logo, stylesheet, chart
if (!assets.stylesheet) {
// Unrecognisable, and it would still be delivered.
throw new RenderBlocked("stylesheet unavailable");
}
if (!invoice.total || !invoice.number) {
// A blank value is a wrong document, not a plainer one.
throw new RenderBlocked("required values missing");
}
const degraded = [];
if (!assets.logo) degraded.push("logo"); // safe to omit
const html = template({ invoice, assets, degraded });
if (degraded.length) {
// Marked, so nobody downstream treats it as complete.
log.warn("degraded document", { invoiceId: invoice.id, degraded });
}
return pdf.post("/v1/pdf", { html, options: { format: "A4" } });
}
/* And prefer deferring: a complete document an hour late beats an
incomplete one now, for almost every document a business sends. */What people do instead
Rendering without the stylesheet when it fails to load, on the grounds that some document is better than none. The result is a complete, unstyled, entirely legible document that gets delivered to a customer, and it is worse than a failure because a failure would have been retried.
What the symptom looks like
A degradation counter rising is a dependency problem rather than a document problem, and it points at asset availability. If it rises without an alert, the degradation is silent, which is the thing to fix first.
Frequently asked
Is this worth doing for a small volume of documents?
Some of it, and the cheap parts are the ones that matter. Allocating a document's identity before rendering it costs nothing and prevents duplicates forever. Logging the template version costs one field and answers most support questions. Queue design, backpressure and capacity planning are genuinely for scale and can wait until there is some.
Why is duplication treated as more serious than latency here?
Because a document usually carries an identity. A duplicated read is harmless and a duplicated invoice is a second numbered document for one event, which somebody has to reconcile by hand and which may already have been sent. That asymmetry is why the correctness half of this cluster is larger than it would be for most APIs.
How does this relate to the troubleshooting pages?
Those start from a symptom you are looking at right now and work back to a cause. These start from a decision made before the symptom exists. The two meet in the middle: a decision skipped here usually appears there some months later as a problem with no obvious explanation.
Related operational topics
The decisions that depend on each other, then the rest of the same group.
What to do when document generation stops working
The decisions to make when rendering is unavailable, most of which are about what to tell people and what to preserve rather than about restoring service.
Alerting on document generation, including failures that return 200
Choosing signals that catch the failures that matter, which for documents includes several that produce no error at all.
Backpressure when documents are requested faster than they render
What the system does when demand exceeds capacity, which is a decision to make rather than a behaviour to discover.
Working out what a generated document actually costs you
Adding up the real per-document cost, which is more than the render and is the number every capacity and caching decision depends on.
Planning capacity for a document peak that arrives once a month
Sizing for a workload that is idle most of the time and enormous occasionally, which is the normal shape for documents and the wrong shape for averages.
Every operational topic
The full list, grouped by correctness, visibility and cost.
What this API actually does
The options and endpoints these decisions are built on, one page each.
Most of these decisions are cheaper to make before the first production run than after the first incident, and none of them need a large system to be worth making.