When an 8 GB pnpm Cache Cost Minutes to Save Seconds

The build had passed. The tests were green. Then the pipeline spent its final minutes uploading an 8.9 GB dependency archive to S3 and failed in the cache step, turning a valid build into a red one. This was the third failure caused by our pnpm CI cache, and the fix was to stop caching the pnpm store at all. A measured restore saved only seconds, while upload time, storage growth, and failure risk cost minutes.

The uncomfortable part was not deleting the cache. It was admitting that my previous remediation had diagnosed the wrong problem.

The prune that fixed nothing

After the pnpm store first grew large enough to break the pipeline during Uploading S3 cache, I reached for the familiar repair:

pnpm store prune

That command removes packages and files no longer referenced by projects known to the store. It is useful when old dependency versions have accumulated after months of lockfile churn.

It also produced reassuring output:

Removed 57316 files (704 MB)
Removed 1224 packages

Then I measured the result:

du -sh "$PNPM_STORE_PATH"

8.0G    /root/.local/share/pnpm/store

The store was still about 8 GB because those bytes were not abandoned debris. They were referenced by the current lockfile. Expo and React Native dependency trees include large native artifacts, and recent over-the-air update work had added more. Pruning had done its job. My assumption had not.

This distinction matters because a cache can grow for two very different reasons:

Treating the second case like the first creates a maintenance ritual without creating a bound. The store can be perfectly clean and still be too expensive to move.

Pnpm CI cache economics beat habit

Caches are usually added with a single expected benefit: faster builds. That is incomplete accounting. A remote CI cache has at least four costs:

1. Time to restore it.
2. Time to upload it.
3. Storage and transfer volume.
4. The probability and impact of cache-step failure.

Our pnpm cache did poorly on that ledger. Restoring roughly 8 GB saved only seconds compared with installing from the package registry. Uploading the store took minutes and had already broken three builds.

A cache hit ratio would not rescue this design. Even a frequent hit is a bad bargain when every run transfers a large archive to avoid a cheap installation step. The useful comparison is end-to-end pipeline time and reliability with the cache enabled versus disabled.

I now use a simple break-even test:

expected benefit = restore frequency × install time saved
expected cost    = restore time + upload time + failure impact

The terms do not need elaborate financial precision. If the restore saves 20 seconds, the upload takes several minutes, and a failed upload invalidates a completed build, the decision is already visible.

Removing the dependency cache

The right change was smaller than the earlier prune logic. We removed the pnpm store from the S3 cache paths and kept dependency installation explicit:

phases:
  install:
    commands:
      - corepack enable
      - pnpm install --frozen-lockfile
  build:
    commands:
      - pnpm run verify
cache:
  paths:
    - /root/.gradle/wrapper/**/*
    - /root/.gradle/caches/modules-2/**/*

The important property is not this particular CI syntax. It is that /root/.local/share/pnpm/store is absent. The lockfile still provides deterministic resolution, and --frozen-lockfile rejects dependency drift. We gave up a few seconds of installation speed in exchange for eliminating a multi-gigabyte upload and one recurring way for a valid build to fail.

There is a cost. Registry availability now matters on every clean runner, and a rate limit or network incident can slow installation. That dependency already existed on cold-cache builds, so the cache had never removed it. It merely made the common path more complicated.

Gradle had the same shape with a slower fuse

Removing pnpm exposed the next habitual cache. On the first green pipeline after the change, the Android cache measured 4.57 GiB. The post-build phase, dominated by cache publication, used 768 seconds of a 60-minute timeout.

It was not failing yet. That was not much comfort. The growth mechanism was similar, but the contents were different.

du -h -d 2 /root/.gradle/caches \
  | sort -h \
  | tail -n 20

Gradle's dependency artifacts are relatively reusable. Its local build cache is different: build-cache-1 gains entries as compilation inputs, plugin versions, build variants, and tasks change. Without an eviction policy or an external size check, retaining the whole directory lets one pipeline run inherit the residue of many earlier ones.

The pnpm lesson did not imply deleting every cache. Gradle compilation can be expensive enough that a bounded build cache earns its keep. The answer was to separate directories by growth mechanism, retain stable downloads, and impose a hard ceiling on generated build entries.

Measure the directories separately

A single total hides the directory responsible for growth. I added an inventory step that reports each major component before publication:

set -eu
 
for path in \
  /root/.gradle/wrapper \
  /root/.gradle/caches/modules-2 \
  /root/.gradle/caches/build-cache-1
do
  if [ -d "$path" ]; then
    du -sh "$path"
  fi
done

This turns cache growth into visible pipeline data. It also prevents another vague diagnosis such as “Gradle is large,” which is true in the same way that “the server is slow” is true.

Put the boundary before upload

We capped the volatile portion before the CI system packaged it. One defensible policy is to discard the local build cache when it exceeds an agreed limit while retaining downloaded modules and the wrapper:

set -eu
 
cache_dir=/root/.gradle/caches/build-cache-1
limit_mib=2048
 
if [ -d "$cache_dir" ]; then
  size_mib=$(du -sm "$cache_dir" | awk '{print $1}')
  if [ "$size_mib" -gt "$limit_mib" ]; then
    echo "Gradle build cache is ${size_mib} MiB; removing it"
    rm -rf -- "$cache_dir"
  fi
fi

Two GiB is an operational choice, not a universal Gradle constant. It should reflect upload bandwidth, build-time savings, and the pipeline timeout. The useful feature is the explicit boundary. A cache that can grow without limit is deferred pipeline work.

This coarse policy also has a sharp edge: crossing the limit produces a cold local build cache on the next run. A native Gradle eviction policy or a dedicated remote build cache can preserve more useful entries, but each introduces configuration and operational ownership. For this pipeline, predictable publication time mattered more than sophisticated retention.

What changed in my review checklist

I no longer approve a new CI cache because the directory is “expensive to recreate.” I ask for measurements from a cold run and a warm run:

That last row deserves attention. Uploading a cache after the build is sensible because the build populates it. Allowing that optional upload to erase a green result is much harder to justify. Where the CI platform permits it, cache publication should be non-fatal or handled outside the critical result path.

The broader point is not that package-manager caches are bad or that Gradle caches are good. Both claims are too blunt to operate a real pipeline. Cache the artifact only when measured reuse pays for measured movement, and set the limit while the directory is still small enough to inspect calmly.

FAQ

Why did pnpm store prune leave an 8 GB store?

pnpm store prune removes unreferenced packages and files. The remaining data was referenced by the current Expo and React Native lockfile, so it was legitimate dependency content rather than stale garbage.

Should I cache the pnpm store in CI?

Only if warm-run measurements show that installation time saved exceeds restore, upload, and failure costs. In this pipeline, restoring the pnpm CI cache saved seconds while uploading it cost minutes, so removing it was cheaper.

How do I find what is making a Gradle cache grow?

Run du -h -d 2 /root/.gradle/caches | sort -h and track major directories independently. Pay particular attention to build-cache-1, which accumulates generated task outputs as build inputs and variants change.

What should happen when a CI cache exceeds its size limit?

Delete or exclude the volatile portion before publication, while retaining stable artifacts that remain economical to restore. The boundary should produce predictable transfer time even if it occasionally causes a cold build.

Can a cache upload failure fail a successful build?

Some CI configurations permit that behavior, but it makes an optional optimization part of the correctness path. A build should be judged by its outputs and verification, not by whether yesterday's shortcut survived one more upload.