Skip to main content

Routing read/classify steps to a cheap model cut cost with no quality loss

Summary: Not every phase needs the frontier model. Reserving the strong model for reasoning/generation and sending the read-and-classify steps to a cheap model kept quality where it mattered and cut cost and latency where it didn't.

What was encounteredโ€‹

Every phase could have gone to the strong model, but several phases only read text and classify โ€” e.g. "did the PM's comment answer question Q1?". Paying frontier prices for that kind of step is pure waste with no quality gain.

How it was solvedโ€‹

Split the pipeline by a single question โ€” is this reasoning, or is this reading? Sonnet (strong) writes SQL: the clarify and draft phases. Haiku (cheap, fast) reads and classifies: preprocess, auto_answer, validate.

Reference: Model routing โ€” a reusable instance of it. Caveat worth remembering: if a "classification" step is actually subtle judgment, the cheap model will quietly get it wrong and poison everything downstream โ€” promote that step back to the strong model.