Deployments and migrationssenior8+ years

A migration adds a nullable column — `ALTER TABLE orders ADD COLUMN gift_note text` — which should be a millisecond metadata-only change on PostgreSQL. Instead, an ordinary `SELECT ... WHERE id = ?` on the same table times out after three seconds. There's an idle transaction from a forgotten reporting query sitting open. Walk through exactly why an unrelated, lock-compatible SELECT gets stuck behind this.

It's not that the SELECT and the ALTER TABLE directly conflict — they don't, an ACCESS SHARE lock (what a SELECT takes) is compatible with the idle session's own ACCESS SHARE lock. The problem is queue order: the ALTER TABLE needs an ACCESS EXCLUSIVE lock, which conflicts with the idle session's ACCESS SHARE, so it queues and waits. Once it's waiting, PostgreSQL will not let a later SELECT jump ahead of it in line, even though that SELECT's lock request would be perfectly compatible with what's currently held — the lock manager checks compatibility against the whole queue in order, not just against what's currently granted, specifically to prevent the ALTER TABLE from waiting forever behind an endless stream of new, individually-compatible reads. So every query on the table piles up behind the migration, which is piled up behind one forgotten idle transaction, and a metadata-only change that should take milliseconds becomes a table-wide outage.

The lesson behind it →