Serving a French audience from one region: what latency taught us

Our public API serves about 40k requests/day, mostly French users, and until last spring it ran out of a single eu-west-3 (Paris) region. p50 looked fine at 180 ms from Paris, but p95 from Marseille and Lyon sat at 620–700 ms, and support tickets from users on mobile in smaller cities kept mentioning “it hangs, then loads.” I assumed this was backend work. It was mostly connection setup: TLS handshake plus DNS, repeated per request because our mobile client wasn’t reusing connections.

First step was measuring, not guessing. I added a small timing header on the edge:

add_header X-Timing-Connect $request_time always;
add_header X-Timing-Upstream $upstream_response_time always;

and logged $remote_addr region by resolving the first two octets against a coarse geo table we already had for VAT checks. That gave us a boring but useful chart: upstream time was flat across regions (~90 ms), while connect time doubled outside Île-de-France. So the fix was client-side and edge-side, not a second region.

What actually helped, in order of impact:

1. Keep-alive on the client. We bumped the HTTP client pool from 1 to 6 idle connections per host and set a 55 s idle timeout, slightly under the load balancer’s 60 s. Mobile retries dropped noticeably.
2. TLS session resumption. We enabled session tickets on the terminator and made sure the client didn’t rotate its session cache per screen. This cut the handshake cost on repeat visits.
3. A tiny edge cache for the two read-heavy endpoints (/v1/regions, /v1/pricing). TTL 300 s, keyed on Accept-Language because the payload is localized. This removed maybe 30% of upstream calls.

# terminator config, trimmed
keepalive_timeout: 60s
session_tickets: on
session_timeout: 12h

Why not add a second region? Cost, mostly. A second region means a second database replica, a failover story we don’t have, and data residency questions we’d have to answer in writing. The measured win from connection reuse was larger than the win we projected from geographic proximity, and it cost us a config change instead of an architecture. The trade-off is real: users on cold connections still pay the handshake, and we accepted that.

Where this breaks down: any client that opens a fresh process per request gains nothing. We saw this in a partner’s server-side integration, which hammered us with new connections and made our p95 look terrible again. Also, session tickets interact badly with some corporate TLS inspectors; we got a handful of errors from one enterprise network and had to fall back to full handshakes there, which we detect by user agent, crudely. And the geo lookup by IP is wrong often enough (VPNs, overseas territories) that I would not use it for anything beyond a chart.

If you’ve served French users from a single region, did you end up going multi-region, or did connection reuse and caching buy you enough time? I’m especially curious whether anyone has measured the actual gain from a second French point of presence rather than assuming it.