The Transcode Worker
Chaos in, H.264 out — and the web server never breaks a sweat.
The incident
Users uploaded video from every device ever made: .mov from iPhones, .mkv from screen recorders, .avi from somewhere in 2011. Browsers could play almost none of it natively. The first fix — transcoding on the web server — worked until two uploads arrived at once and the CPU pegged, taking the whole site down with it.
Why the obvious approach fails
Transcoding is one of the most CPU-hungry things a server can do, and upload traffic is bursty by nature — the worst possible pairing for the machines that also serve your pages. Doing it synchronously in the request path compounds the error: the user's connection is held open for minutes while FFmpeg grinds, and a timeout means starting over. Media conversion is batch work wearing a real-time costume. It belongs in a decoupled worker fleet, sized and scaled independently, with the user free the moment the raw bytes land.
The standard
- Upload and processing are fully decoupled: raw files land (via the Blob Ingestor), an event enters the job queue, and the user's request completes immediately.
- A dedicated FFmpeg worker fleet normalizes everything to H.264/AAC — universal playability, one pipeline.
- Outputs include the adaptive ladder (feeding the Adaptive Stream) and auto-extracted thumbnails and scrub-bar sprites.
- Failed jobs retry with backoff and land in a dead-letter queue with the source preserved — a bad file never silently vanishes.
- Processing latency (upload-to-playable) is a tracked SLO per file-size class.
How Cortex validates it
At plan time
media-accepting blueprints declare the pipeline stages, target formats, and processing SLOs.
At validation
assert: request_path.transcode_work == 0assert: outputs.format == h264_aac && ladder_generatedassert: failed_jobs → dlq(source_preserved)Assertion syntax is illustrative.
At review
the approver sees the chaos-corpus run: a folder of hostile formats in, playable normalized outputs with thumbnails out, and the worker fleet's CPU isolated from the web tier throughout.