Testing what happens when a render fails, not just when it works

Why the obvious approach does not work

Document suites test the happy path because that is what a fixture produces. The failure branches, a render that times out, a payload that is rejected, a request that is rate limited, a stored document that expired, are written once and never executed, so they contain the usual crop of defects: an error swallowed, a retry that duplicates, a user-facing message that leaks an internal detail, a partial file written to disk. Those branches are cheap to test because they need no rendering at all, only a client that returns the error.

What to do instead

Each of these is a decision with a wrong answer, and the wrong answer usually produces a suite that passes rather than one that fails.

• Test against the error shapes the API actually returns, by stubbing the client rather than by trying to provoke real failures.
• Assert that a failed render does not produce a file. A truncated or empty PDF written to storage is worse than no PDF, because something downstream will serve it.
• Assert the user-facing message on failure is the one you intend, and that it does not contain an internal identifier, a stack trace or a URL.
• Test the retry path explicitly, including that a retry after a partial success does not produce a second document.
• Test the rate-limited branch, since it is the one most likely to fire in production and least likely to have been run in development. A 429 carries a Retry-After header, and the test should assert your client honours it rather than retrying immediately.
• Test the expiry branch for stored documents, since a retrieval after expiry is a normal event rather than an exceptional one.

In practice

Test code, with the thing people write instead kept in a comment where it is the more instructive half.

// No rendering required: stub the client and exercise the branches.
test("a failed render writes no file", async () => {
  client.post.mockRejectedValue(new ApiError(503, "renderer_unavailable"));

  await expect(generateInvoice(invoice)).rejects.toThrow(RenderFailed);
  expect(await storage.exists(`invoices/${invoice.id}.pdf`)).toBe(false);
});

test("the message shown to a user leaks nothing", async () => {
  client.post.mockRejectedValue(new ApiError(500, "db_error", "conn=10.0.3.4"));

  const shown = await renderPageFor(invoice).catch((e) => e.userMessage);
  expect(shown).toBe("We could not produce that document. Please try again.");
  expect(shown).not.toMatch(/10\.0\.|stack|Error:|https?:/);
});

test("a rate limit is honoured rather than hammered", async () => {
  client.post
    .mockRejectedValueOnce(new ApiError(429, "rate_limited", null, { "retry-after": "30" }))
    .mockResolvedValueOnce({ body: pdfBytes });

  const sleep = jest.spyOn(timers, "sleep").mockResolvedValue(undefined);
  await generateInvoice(invoice);

  expect(sleep).toHaveBeenCalledWith(30_000);   // the header, not a guess
  expect(client.post).toHaveBeenCalledTimes(2);
});

test("retrieving an expired document is handled, not thrown at a user", async () => {
  client.get.mockRejectedValue(new ApiError(410, "content_expired"));
  await expect(fetchStored(docId)).resolves.toEqual({ expired: true });
});

What people do instead

Testing only that an error is thrown. What matters is what the system does around the error: whether a file was written, whether a unit was charged, whether a retry will duplicate, and what the user sees. An assertion that a promise rejected covers none of that.

What a failure actually tells you

These tests fail when somebody changes error handling without meaning to, which is the most common way a careful failure path degrades into a swallowed exception over a few refactors.

Frequently asked

Related testing techniques

The techniques that pair with this one, then the rest of the same group.