The Zero-Downtime Migrator

The incident

The migration was one line: add a NOT NULL column with a default. On the staging database it took a second. On the production orders table — two hundred million rows — it took an exclusive lock and eleven minutes, during which every order in the company queued behind an ALTER TABLE. The system that had promised zero downtime delivered eleven minutes of it, from one line everyone had approved.

Why the obvious approach fails

Schema changes that are instant on small tables can be lock-holding sieges on large ones — and the difference is invisible in every environment except the one that matters. The deeper trap is coupling: changing the schema and the code in one step means there's a moment when they must match exactly, and deploys aren't instantaneous across a fleet. Live migration is a choreography problem: expand the schema first (additive, non-locking), migrate data asynchronously in batches, run dual writes while both shapes coexist, verify parity, shift reads, and only then contract — with every step independently deployable and independently reversible.

The standard

• +Migrations follow expand-then-contract: additive changes first, destructive changes last, and never in the same release as the code that depends on them.
• +Large-table operations are lock-audited before they run: anything taking more than a declared lock budget must run as an online/batched operation.
• +Data backfills run asynchronously in bounded batches with progress tracking — never as one transaction holding one lock.
• +Dual-write windows keep old and new shapes consistent during transition, with parity verification before reads shift.
• +Every migration ships with its reverse; 'roll forward only' requires explicit sign-off, not habit.

Composes with