The APCu Cache That Survived a Deploy and Broke Pricing

PHP 8.1 on three FPM pools behind an ALB, Debian 11, APCu 5.1.21 with apc.enable_cli=0. We cache a per-currency pricing table built from a 12k-row product matrix; the array is roughly 4MB serialized, 18MB after unserialize. It lives in APCu under a key that includes a schema version, and we warm it on the first request after deploy. For about six weeks, every second deploy left one or two hosts serving old prices. Not all hosts, not always, and never in staging, because staging has a single pool and deploys during a quiet window.

The first thing I stopped trusting was the deploy script. It called apcu_clear_cache() over a local PHP CLI invocation, which with apc.enable_cli=0 clears a different, empty cache and exits 0. So the web pool’s cache was never touched. I replaced that with an FPM endpoint hit over loopback, but only after checking it actually reaches the pool:

if (!apcu_enabled()) {
    error_log('apcu disabled in this pool');
    http_response_code(503);
    exit;
}
apcu_clear_cache();
echo json_encode(['cleared' => true, 'pid' => getmypid()]);

That fixed the explicit clears. The remaining misses came from the warm path: two concurrent requests both saw a miss and both rebuilt the table, and the second write could land after a request had already read the first. We now hold a short lock and re-check inside it:

$lock = 'pricing_rebuild_lock';
if (apcu_add($lock, 1, 30)) {
    try {
        if (apcu_fetch($key, $ok) === false || !$ok) {
            apcu_store($key, build_pricing_table(), 3600);
        }
    } finally {
        apcu_delete($lock);
    }
}

The mechanism is boring: APCu is per-process-pool shared memory, so “clear the cache” is only meaningful if it runs inside the pool you care about. A CLI call is a different address space with its own cache. The lock costs one extra apcu_add per miss and can leave a 30-second stale lock if the process dies mid-build; we accept that because the rebuild is idempotent and the TTL bounds the damage.

Where this falls apart: it does nothing for multi-host consistency, and it assumes the rebuild is fast enough that a 30s lock is generous. I have seen a 40s rebuild under cold IO hold the lock, let the next request through unlocked, and rebuild again; two concurrent builds, one lost write. If the table ever outgrows APCu’s apc.shm_size, the store fails silently and you rebuild every request. Check apcu_store()’s return value, not just the absence of an error.

What do you use to verify a cache clear actually hit every pool after a deploy, without adding a full shared cache?