Multi-Model QA Cascade — Claude-Orchestrated, 3-Provider, $0 ★ CENTERPIECE

Code: 02_ai-engineering-code/multimodel-qa-cascade/ (rewrite-qa.mjs, run.sh, test-providers.sh, README.md, output/sample-one-item.json)

Origin: tradetesttwo/manual/2026-03-03-qa-rewrite-pipeline/

Problem. 2,760+ flagged defects across exam questions, answer explanations, learning assets, and flashcards needed rewriting — fast, cheap, and without trusting any single model's word for "good."

What I built. A content-rewrite engine, run under Claude Code as orchestrator, that fans every item out to three providers in parallel:

Ollama (local, llama3.x via localhost:11434) — zero-cost local baseline.
Groq (llama-3.3-70b, free tier) — fast cloud pass.
OpenAI Codex (gpt-5.x via OAuth codex exec) — highest-quality pass.

Total spend: $0 (local + free tiers + login-based, no per-call billing).

The key idea — verify deterministically, decide with a human.

Every provider's output is scored by category-specific validators that are pure regex + structure checks, not an LLM (e.g. question options = 100 pts across structure/word-count/balance/no-meta-commentary/unique-options; pass ≥ 70). Same input → same score, always.
It's a comparison engine, not cascade-and-stop: the reviewer sees the original + all three rewrites side-by-side with per-check breakdowns, picks the best, and the tool emits SQL patches (apply-*.sql) using the highest-scoring version.
Production-grade ops: parallel fan-out, retry, --resume, --dry-run, --skip-ollama, rate-limit handling, preflight test-providers.sh.

Why it's the centerpiece for an AI-native role. It's the whole job in one artifact: orchestrating heterogeneous models by cost/speed/quality, refusing to let a model grade itself (deterministic validators), keeping a human in the decision loop, and shipping the result as executable patches.

GG · Applied AI Engineering portfolio · synced to youdontneedmy.help design