What I learned moving a French team's on-call rotation to CET-aware tooling
这篇内容暂时只有 English 版本,以下是原文。
Our team of six sits in Paris and Lyon; the services run in eu-west-3. For two years the on-call schedule lived in a spreadsheet with a note that said “heures locales” and that was the whole timezone strategy. PagerDuty was configured with a single schedule and the rotation handoff at 09:00 Europe/Paris. It worked until it didn’t: after the March DST switch, the handoff landed at 08:00 for the Lyon people on the Monday after the change, and the person who was supposed to be off kept getting pages for two hours. Nobody noticed for a week because alerts were quiet.
The fix was not glamorous. First, I stopped treating the schedule as a human artifact and made the rotation a generated file. The source of truth is a small Python script that emits the schedule for the next quarter:
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
paris = ZoneInfo("Europe/Paris")
start = datetime(2025, 1, 6, 9, 0, tzinfo=paris)
for week in range(13):
handoff = start + timedelta(weeks=week)
print(handoff.isoformat(), handoff.astimezone(ZoneInfo("UTC")).isoformat())
Second, every alert route and escalation policy in the tool is keyed to UTC, and the only place Europe/Paris appears is in the display layer and the handoff definition. Third, I added a test that asserts the number of hours between consecutive handoffs is exactly 168 even across the March and October switches. That test caught the second problem before the next switch: one of our maintenance windows was defined as “02:30 local every Sunday” and in October that time occurs twice.
Why this shape: DST is a presentation problem, not a scheduling problem. Once handoffs are absolute instants, the only remaining risk is code that re-derives local time from a wall-clock string. The cost is that humans now read UTC in the tool and have to translate; I accepted that because the alternative was a class of bug that only appears twice a year and is invisible the rest of the time. I also gave up on the spreadsheet, which two people still miss.
Where it breaks: anything that fires on a calendar-day boundary. Our daily digest job is scheduled in the tool as “08:00 Europe/Paris” and that is fine, but the metrics dashboard behind it buckets by UTC day, so the digest and the dashboard disagree for two hours every morning. I have not fixed that. Also, if you have colleagues in Nouméa or Cayenne, a single “local” handoff assumption is simply wrong; you need per-person timezones and the test has to check each one.
One thing I am still unsure about: whether it is better to show local time in the paging UI and accept the ambiguity, or show UTC and accept the friction. We chose UTC for the schedule and local for the alert timestamps, which is inconsistent. What do other teams with distributed French sites do for the handoff display?
0 条评论
还没有评论,写下第一条。