PDFPipe

Container orchestration / Kubernetes

Puppeteer on Kubernetes: OOMKilled with no stack trace

The pod restarts. There is no application error, no exception, and the last log line is whatever was printed before the render started. The pod status says OOMKilled.

What is actually wrong

Two things at once, and they are commonly confused. The 64 MB /dev/shm default crashes browser tabs under concurrency, which surfaces as protocol errors. Separately, Chromium's memory is not your runtime's heap: a container limit sized from Node's heap usage does not account for a browser process tree, so the kernel out-of-memory killer terminates the container. It kills the process rather than raising an error, so nothing in the application ever sees it.

What Kubernetes gives you to work with

the same container as Docker, scheduled onto a node with a memory limit the kernel enforces by killing things.

  • Packaging: no image size limit, but a large image slows every scale-up, because a new pod on a cold node waits for the pull.
  • Filesystem: as configured. /dev/shm defaults to 64 MB and is the single most common cause of browser crashes in a cluster.
  • Processes: full control inside the pod. Memory limits are enforced by the kernel out-of-memory killer, which terminates the process without a stack trace and without an application-level error.

The fix, if you are keeping Puppeteer

Mount /dev/shm as a memory-backed volume of a useful size, set the container memory limit from observed browser usage rather than from runtime heap, and add an init process so orphaned renderers are reaped instead of accumulating against the pod's process limit.

yaml
spec:
  # An init that reaps the browser's orphaned children.
  shareProcessNamespace: true
  containers:
    - name: renderer
      resources:
        limits:
          # Sized from browser usage under load, not from heap.
          memory: "2Gi"
      volumeMounts:
        - name: dshm
          mountPath: /dev/shm
  volumes:
    - name: dshm
      emptyDir:
        medium: Memory
        sizeLimit: 1Gi

What still hurts after that

A memory-backed emptyDir counts against the pod's memory, so raising the shared memory mount lowers what is left for everything else and the two numbers have to be tuned together. Horizontal autoscaling on CPU behaves poorly here because a render is bursty: the pod is idle, then saturated for two seconds, then idle. And every replica now reserves browser-sized memory whether or not it is rendering, which is the cost of putting a browser in a long-lived pod.

What Puppeteer is genuinely good at

it renders exactly what a browser renders, which means modern CSS, web fonts and client-side JavaScript all behave the way they do in a tab. That is worth keeping in view: the problem on this page is where it runs, not what it produces. Puppeteer is a Node library that drives a real Chromium, and downloads that Chromium into node_modules when it is installed. It needs the browser binary, a long list of shared libraries the browser links against, fonts, and enough memory for a browser process that is not accounted for by your runtime's heap settings.

The other shape of the answer

Every fix above keeps the renderer inside your deployment, which means the constraint stays yours: the package size, the shared libraries, the fonts, the memory, the patching. The alternative is an HTTP call from Kubernetes to something that already has a browser in it, which turns all of the above into a request and a response. That is what this API is, and it is the reason these pages exist. Whether that trade is right depends on whether rendering is a thing you want to operate.

Frequently asked

Can Puppeteer run on Kubernetes at all?

Mount /dev/shm as a memory-backed volume of a useful size, set the container memory limit from observed browser usage rather than from runtime heap, and add an init process so orphaned renderers are reaped instead of accumulating against the pod's process limit. Whether that is worth doing is a separate question from whether it is possible, and the section above on what still hurts is the honest answer to it.

Why does it work locally?

Because a development machine has root, a package manager, several gigabytes of memory, a full font set and no packaging limit. Every one of those is a thing Kubernetes constrains. A renderer that works locally and fails on deploy has not regressed: it has met the first environment that does not give it everything.

Related failures

The same platform with a different renderer, and the same renderer on other platforms.

If the render moves off this platform, none of the above is your problem to keep solving. 100 free documents a month, and a playground that needs no signup.