Rollback and Migration Safety

How It Works

Rollbacks are often not simple, especially with stateful databases. This recipe introduces

a safer change model: backward-compatible migrations, feature flags, and practiced recovery.

Triggers

• Rollbacks frequently fail or require "surgery"
• Database migrations cause late-night incidents
• Change failure rate and MTTR are high

Steps

1. Prefer roll-forward fixes by making pipelines fast and reliable.
2. Make schema changes backward compatible:

• add new columns first,
• deploy code that supports both,
• migrate data,
• only then remove old fields.

1. Separate schema deploy from feature exposure using flags where feasible.
2. Add a migration rehearsal in staging with production-like data volume assumptions.
3. Document and test recovery steps regularly (tabletop or practice run).

Expected Outcome

• Fewer catastrophic rollout failures and faster recovery when failures occur.
• Migration failures become predictable events with known mitigation.

Example Inputs

• "We need zero-downtime migration strategy."
• "Rollback didn't work last time; we need a safer approach."
• "How do we keep code compatible with one DB change ahead?"

Tips

• If rollback is your plan, ensure you can actually execute it under pressure.