Forty years in software has taught me one thing that never changes: the bill always arrives eventually. Right now a lot of SaaS founders are baking LLM calls into every feature they ship, and the bill for that is quietly becoming the biggest line item on the P&L. So it's worth paying attention when a company like Writer announces a new model and harness built specifically to contain token costs, rather than just chase another benchmark leaderboard.
The bit everyone glosses over
Every launch post for the last two years has led with capability. Smarter reasoning, longer context, better code. All true, all useful, and all beside the point if you're the person actually running these things in production at volume. What I care about — what anyone running RSSMasher or Article2Video pipelines across thousands of jobs a day cares about — is what it costs to get a good answer, repeatedly, forever.
Writer built their new model on an open-source base specifically to cut inference costs, and paired it with an "upgraded harness" — their term for the orchestration layer that decides how and when the model actually gets called. That's the interesting part. Not the model. The harness.
Why the harness matters more than the model
Most people optimising AI costs go straight for the obvious lever: swap to a cheaper model, quantise it, fine-tune something smaller. All valid. But the bigger wins, in my experience building content-automation tools, come from not calling the model at all when you don't need to.
A well-built harness will:
- Cache and reuse outputs for near-identical inputs instead of regenerating from scratch
- Route simple tasks to a cheap, fast model and reserve the expensive one for genuinely hard steps
- Batch and compress prompts so you're not paying to re-send the same boilerplate instructions on every call
- Fail fast and retry cheaply rather than burning tokens on a full regeneration when a small correction would do
That's not clever AI research. That's just good engineering discipline, applied to a new kind of expensive resource. Tokens are the new API rate limit, and anyone who lived through the early days of paying per API call already knows the playbook: cache aggressively, batch sensibly, and never call out for something you already have sitting in storage.
What this means if you're building on top of LLMs
If you're running any kind of content pipeline — video generation, article rewriting, marketing copy at scale — token cost isn't a rounding error once you're past a few hundred users. It's the difference between a healthy margin and a business that's technically growing but bleeding cash on every job.
A few things worth actually doing this week, not just nodding along to:
Audit your call patterns. Pull a week of logs and work out which steps in your pipeline are making redundant calls — same input, same output, paid for twice. This is nearly always a bigger saving than switching models.
Tier your model usage. Not every step needs your best (most expensive) model. Classification, tagging, simple reformatting — send that to something cheap. Save the expensive calls for the steps where quality genuinely moves the needle for the end user.
Treat your harness as a product, not plumbing. Writer clearly spent real engineering effort on the orchestration layer, not just the model weights. That tells you where the industry's cost pressure is actually going to be solved — not in bigger models, but in smarter systems around smaller ones.
The alchemy angle
The whole premise of what I build is turning raw content into something valuable — that's the trade. But alchemy was never really about magic, it was about process discipline: knowing exactly what goes in, what comes out, and not wasting the expensive stuff on steps that don't need it. Writer's move is a reminder that the AI industry is growing up. The novelty phase, where nobody counted the cost of a token because the demos were too exciting, is ending. The efficiency phase is starting, and it's going to separate the tools that scale profitably from the ones that just scale.
If you're running LLM pipelines at any real volume, go and read your own token logs before you read another model benchmark. The benchmark won't save your margin. The audit might.
— Wayne