The Saga Conductor
Across services, every step knows how to undo itself.
The incident
The order service committed. The payment service committed. The inventory service threw a timeout. The system settled into a state no one had designed and no one could name: paid, confirmed, and never going to ship. Finding these orders took a SQL query someone wrote at midnight. Fixing them took a week of manual reconciliation, one apologetic email at a time.
Why the obvious approach fails
Inside one database, transactions give you all-or-nothing for free. Across services, that guarantee evaporates — there's no distributed rollback, and pretending otherwise (two-phase commit across the internet) trades correctness problems for availability ones. Partial failure isn't an edge case in a distributed system; it's the weather. The saga pattern accepts this and designs for it: a multi-service operation is an explicit sequence of local steps, each paired with a compensation that undoes it. When step four fails, steps three, two, and one compensate in reverse — automatically, by design, into a state somebody actually intended.
The standard
- Every multi-service business operation is modeled as an explicit saga — named steps, declared order, defined outcome states.
- Every step ships with its compensation; a step without an undo doesn't pass design review.
- Saga state is persisted: an orchestrator (or well-defined choreography) knows where every in-flight saga stands, across restarts.
- Compensations are idempotent and retried with the same discipline as forward steps — an undo that can fail silently is not an undo.
- Stuck sagas are visible: anything in-flight past its budget alerts with full step history, so 'paid but never shipped' is a monitored state, not a midnight query.
How Cortex validates it
At plan time
cross-service Missions map every operation as a saga with per-step compensations before build.
At validation
assert: multi_service_ops.modeled_as == sagaassert: steps.compensation_defined == 100%assert: fail(step_n) → compensated(steps 1..n−1) && terminal_state ∈ designed_statesAssertion syntax is illustrative.
At review
the approver sees the failure-injection matrix: each step failed on purpose, the compensation chain executing, and every terminal state one that was designed.