Measure Django Cache Performance Before Adding More Caching

At 4:30 Friday, one unanswered question halted our approval of application-wide Django caching: Which slow requests would better cache performance actually fix? Our production evidence could not identify a single one.

We had requests completing in under roughly 500 ms, reports of slower paths, prior downtime, a database-backed cache record, and signal-driven invalidation. Those observations made caching relevant. They did not show that misses caused the reported latency or that broader caching would improve it.

Expansion also carries operating costs: memory pressure, cold-start exposure, key migrations, stale values, invalidation fan-out, and additional policy to maintain. The named failure mode was ineffective cache expansion: paying those costs without removing the work that limited slow requests.

Find the work that limits cache performance

The reusable sequence is simple: match comparable requests, identify their limiting work, verify that a hit removes it, and estimate the policy-level effect experimentally. Each step answers a different question. Skipping one turns observation into an unsupported causal claim.

The same request duration can hide different mechanisms. A miss may wait on a database query. Hits and misses may both wait on the same remote call. A hit can remain expensive because of decoding, rendering, pool contention, or lock wait. Frequent invalidation may prevent reuse.

Cache hit rate is an implementation metric. Avoided limiting work is the causal mechanism that matters. Connect each stable route trace to its cache operations, dependent database or external spans, and relevant invalidation activity.

Raw URLs, object identifiers, users, and cache keys are unsafe telemetry dimensions. Use resolved route names and bounded operation names. Put detailed identifiers only in sampled diagnostic logs when incident controls permit them.

Keep bounded counters and histograms unsampled. State trace sampling rates. Sampled traces can explain mechanisms, but they cannot establish an exact fleet-wide hit ratio.

The official Django cache documentation remains the authority for backend configuration, timeouts, key behavior, and API semantics.

Instrument each cache stage honestly

A request-level variable is wrong when one request performs several cache operations because later operations overwrite earlier ones. Create separate cache.get, loader, and cache.set spans and timers for every bounded operation. Do not label the duration of loading and writing as lookup time.

A private sentinel distinguishes a miss from a cached null. Every stage records its outcome and duration even when it raises. The example below treats a failed cache write as an availability problem, not a failed load: it records the error and returns the loaded value. Use fail-closed behavior only when a stated correctness or security invariant requires the value to be stored before it can be returned.

import time
 
_MISSING = object()
 
 
def load_cached(cache, key, loader, tracer, operation, ttl_seconds, metrics):
    get_started = time.perf_counter()
    with tracer.start_span("cache.get", {"cache.operation": operation}) as span:
        try:
            value = cache.get(key, _MISSING)
            get_outcome = "hit" if value is not _MISSING else "miss"
        except Exception:
            get_outcome = "error"
            raise
        finally:
            elapsed = time.perf_counter() - get_started
            span.set_attribute("cache.get_outcome", get_outcome)
            metrics.observe("cache.get_seconds", elapsed, operation, get_outcome)
 
    if value is not _MISSING:
        return value
    return _load_and_store(
        cache, key, loader, tracer, operation, ttl_seconds, metrics
    )
 
 
def _load_and_store(cache, key, loader, tracer, operation, ttl_seconds, metrics):
    load_started = time.perf_counter()
    with tracer.start_span("cache.loader", {"cache.operation": operation}) as span:
        try:
            value = loader()
            load_outcome = "success"
        except Exception:
            load_outcome = "error"
            raise
        finally:
            elapsed = time.perf_counter() - load_started
            span.set_attribute("cache.loader_outcome", load_outcome)
            metrics.observe("cache.loader_seconds", elapsed, operation, load_outcome)
 
    set_started = time.perf_counter()
    with tracer.start_span("cache.set", {"cache.operation": operation}) as span:
        try:
            stored = cache.set(key, value, timeout=ttl_seconds)
            set_outcome = "rejected" if stored is False else "success"
            if stored is False:
                metrics.increment("cache.set_errors", operation)
        except Exception as exc:
            set_outcome = "error"
            span.record_exception(exc)
            metrics.increment("cache.set_errors", operation)
            # Default availability policy: return the successfully loaded value.
        finally:
            elapsed = time.perf_counter() - set_started
            span.set_attribute("cache.set_outcome", set_outcome)
            metrics.observe("cache.set_seconds", elapsed, operation, set_outcome)
    return value

This pseudocode assumes the tracing and metrics implementations tolerate recording in finally blocks. Test five paths explicitly: a cached null, a cache-read exception, a loader exception, a cache-write exception, and a backend that reports write rejection by return value. The default policy returns a successfully loaded value after write failure; a correctness or security invariant may require a documented fail-closed policy instead.

Summarize a request only after all child spans finish as all_hits, all_misses, mixed, or error. Exclude requests without cache operations from that comparison.

Compare matched cohorts and tail distributions

Do not compare every hit with every miss. They often involve different routes, operations, key popularity, request shapes, cache ages, payload sizes, or traffic periods.

Define cohorts before reading results. Match on stable route and cache operation, then control attributes that can change limiting work, including request variant, response shape, dependency state, and load level. Compare bounded key-shape or popularity buckets rather than identifiers. Record cache age when the backend permits it, or derive bounded age bands from controlled key creation. If age is unavailable, state that limitation.

For each cohort, inspect the latency distribution, including upper-tail behavior, rather than averages alone. Then inspect representative traces. Determine whether hits omit the database query, remote call, loader computation, or coordination wait present on misses. This is mechanistic evidence of association, not yet proof of production-wide benefit.

A cache stampede is a distinct failure path. After cold start or invalidation, concurrent requests can load the same popular entry and multiply database or external work. Longer TTLs move the burst without removing it. Capture loader concurrency and coordination wait. Test finite-lease locking, single-flight coordination, or stale serving with an explicit owner-failure path.

Estimate policy-level effect experimentally

Use a controlled rollout or representative load test instead of comparing unrelated before-and-after windows. Randomly assign eligible traffic, keys, or another defensible unit where interference is manageable. Keep route mix, offered load, cache warmness, key-shape distribution, dependency conditions, and observation windows comparable. If randomization is impossible, use matched control traffic and document the remaining confounders.

Report request latency distributions, errors, loader work, concurrent loads, invalidations, and resource consumption with identical definitions. Check cold and warm periods. Stop or narrow exposure if correctness or reliability signals regress.

The unresolved proof boundary is the production effect under representative load. Traces can show that a hit coincides with removed work. Only the controlled comparison estimates whether the proposed policy improves latency, subject to its controls and interference limits. The production performance measurement chapter covers deployment windows and confounders in more detail.

Invalidate only after commit

Signal-driven invalidation can run before its transaction commits. A reader may then miss, reload old database state, and leave that value cached after commit. Register correctness-sensitive deletion with transaction.on_commit(). Record a bounded reason and affected-entry count, and inspect bulk mutations for invalidation churn and refill pressure.

Telemetry does not remove signal coupling. Receiver ordering, bulk-operation behavior, discoverability, and test coverage remain engineering concerns.

FAQ

What if a cache get fails?

Record cache.get_outcome=error, preserve exception-safe timing, and state whether the application fails open or closed. Calling an error a miss hides a reliability dependency.

Should a failed cache write fail the request?

Normally, no. Return the successfully loaded value and record the failed write. Fail closed only when a documented correctness or security invariant requires successful storage before response.

What if null should not be cached?

Keep the sentinel so null and miss remain distinct, then make null caching explicit. Any negative-cache TTL creates a freshness edge after creation.