Operations / Correct under repetition
Idempotency for a document that must not be generated twice
Making a repeated request produce the same document rather than a second one, which matters more here than in most APIs because the artefact is often numbered.
Why the default answer is wrong here
Most idempotency discussions are about not charging a card twice. A document has the same shape of problem with an extra wrinkle: the second document is frequently not identical to the first, because generating it consumed a number. An invoice sequence advances, a reference is allocated, a timestamp moves. So a duplicate is not a harmless repeat, it is a second document with a different identity for the same underlying event, and reconciling that afterwards is manual work.
The decisions
Reasons rather than a description of the code. Each has a default that is defensible in general and wrong for documents specifically.
- Allocate the identity before the render, not during it. The invoice number, the reference and the issue date are decided by your system and stored, and the render is a pure function of them.
- Key the operation on something from your own domain, an order identifier or an event identifier, rather than on a generated request identifier that changes when the caller retries.
- Record the outcome against that key before returning it, so a repeat finds the previous result rather than starting again.
- Store the produced document rather than only its bytes in a response, so a repeat can return the same file instead of re-rendering and hoping it matches.
- Decide explicitly what a repeat with different data should do. Usually it is an error, because it means the caller believes something changed and the document already went out.
- Remember the render itself is not the risky part. The risky part is the identity allocation and the delivery on either side of it.
In practice
A fragment, with the thing that goes wrong kept in a comment where it is the more instructive half.
// The identity is allocated once, by us, and stored. The render is a
// pure function of it, so a repeat produces the same document.
async function generateInvoice(orderId) {
const existing = await db.invoices.findByOrderId(orderId);
if (existing?.documentId) {
return { documentId: existing.documentId, reused: true };
}
// Allocate identity first, in a transaction, keyed on our own domain.
const invoice = existing ?? await db.transaction(async (tx) => {
const number = await tx.sequences.next("invoice");
return tx.invoices.create({ orderId, number, issuedAt: new Date() });
});
const res = await pdf.post("/v1/pdf", {
html: renderInvoice(invoice),
filename: `invoice-${invoice.number}.pdf`,
store: true,
});
await db.invoices.update(invoice.id, {
documentId: res.document_id,
documentExpires: res.document_expires,
});
return { documentId: res.document_id, reused: false };
}
/* Why the identity comes first: if the render is retried after a
timeout, the second attempt reuses the same number and produces the
same document. If the number were allocated during the render, the
retry would produce invoice 119 for an order that already has 118. */What people do instead
Keying idempotency on a request identifier the caller generates. A caller retrying after a network timeout usually generates a fresh one, so the key differs, and the mechanism that exists to prevent duplication is bypassed by the exact scenario it was built for.
What the symptom looks like
Two documents with different numbers for the same order is the symptom, and it appears in reconciliation rather than in monitoring. If it has happened once it will have happened more, so it is worth counting rather than fixing the one that was reported.
Frequently asked
Is this worth doing for a small volume of documents?
Some of it, and the cheap parts are the ones that matter. Allocating a document's identity before rendering it costs nothing and prevents duplicates forever. Logging the template version costs one field and answers most support questions. Queue design, backpressure and capacity planning are genuinely for scale and can wait until there is some.
Why is duplication treated as more serious than latency here?
Because a document usually carries an identity. A duplicated read is harmless and a duplicated invoice is a second numbered document for one event, which somebody has to reconcile by hand and which may already have been sent. That asymmetry is why the correctness half of this cluster is larger than it would be for most APIs.
How does this relate to the troubleshooting pages?
Those start from a symptom you are looking at right now and work back to a cause. These start from a decision made before the symptom exists. The two meet in the middle: a decision skipped here usually appears there some months later as a problem with no obvious explanation.
Related operational topics
The decisions that depend on each other, then the rest of the same group.
Retrying a render safely, and which failures are safe to retry
Deciding which failures should be retried and which should not, because a blanket retry on a document endpoint is a duplication mechanism.
Stopping the same document being requested twice at once
Collapsing concurrent identical requests, which is a different problem from idempotency because both arrive before either has finished.
Queue design for a payroll run or any large document batch
Structuring a job that produces thousands of documents at once, where the interesting decisions are about failure and progress rather than throughput.
Caching a rendered document, and when it is safe to serve one
Serving a previously rendered document instead of making a new one, which is nearly free when the inputs are unchanged and wrong when they are not.
What belongs in the cache key for a rendered document
The full list of inputs a rendered document depends on, which is longer than the data and is the reason document caches serve stale output.
Every operational topic
The full list, grouped by correctness, visibility and cost.
What this API actually does
The options and endpoints these decisions are built on, one page each.
Most of these decisions are cheaper to make before the first production run than after the first incident, and none of them need a large system to be worth making.