Engineering Intelligence. Read the book.
module-35/vision-processor.mdPRODUCTION MODULE

The Vision Processor

User-uploaded images made searchable, safe, and private — before anyone sees them.

Thousands of user-uploaded receipts and IDs, processed by hand — with faces and ID numbers sitting unredacted in a bucket.
An async pipeline that enhances, OCRs, and privacy-redacts imagery before a human or model ever sees it.

The incident

Thousands of KYC documents and receipts, uploaded daily, processed by a human queue. Ops drowned; turnaround stretched to days. Worse: raw images — faces, ID numbers, addresses — sat unredacted in a bucket with broad internal read access. Two problems, one root cause: no intake pipeline.

Why the obvious approach fails

Treating uploaded images as opaque blobs defers every problem to a human: extraction (someone reads it), quality (someone squints at it), privacy (someone remembers to care). None of that scales, and the privacy failure is a breach in waiting — regulated identifiers shouldn't exist unredacted at rest just because they arrived as pixels instead of text. Media intake is a processing problem: normalize, extract, classify, and redact asynchronously, so what lands in storage is already safe and already useful.

The standard

  • Uploads enter an async pipeline (via the Job Marshall pattern) — no synchronous processing in the request path, no raw image required for downstream work.
  • Pre-processing normalizes what OCR needs: rotation, contrast, de-skew.
  • OCR extraction turns documents into indexed, searchable text tied to the source image.
  • PII detection runs before storage finalizes: faces and identifier regions are redacted or vaulted per data classification; broad-access storage never holds raw identifiers.
  • Every processing step is traceable per asset — what was extracted, what was redacted, and why.

How Cortex validates it

At plan time

Any blueprint accepting user media declares the pipeline stages and the data classification of expected content.

At validation

assert: request_path.image_processing == none (async only)assert: finalized_storage.unredacted_pii == 0 on seeded corpusassert: extracted_text.indexed && linked(source_asset)

Assertion syntax is illustrative.

At review

The approver sees the seeded-corpus run: sample IDs and receipts in, redacted assets and searchable text out, with the per-asset trace.

Composes with

The Shadow SentinelThe Blob Ingestor patternThe Job Marshall pattern

See where your codebase stands against this standard.

Get an Agent Readiness AssessmentBack to the library