Compounding systems vs. one-off automations
One-off automations save hours. Compounding systems change the slope of the operation. The cost looks the same, the return does not.
Two systems, same upfront cost. Same model, same vendor, same six-figure check. Twelve months later, one is generating ten times the value the other is. The difference isn't quality. It's whether the system was built to compound.
A one-off automation runs the same logic on inputs forever. You wire it up, it does the thing, you move on. It saves real hours, but only the hours it was built to save. The day after launch is the best day it will ever have.
A compounding system improves with traffic. Each resolution produces a signal. Each signal updates the model, the rubric, the routing rule, or all three. The system in week 12 is a different system than the one in week 4, and a different one again in month 6. The team sees the slope, not just the level.
The trait that separates them is the feedback chain. Three pieces:
Telemetry
Every decision the system makes is logged with enough context to grade it later. Inputs, outputs, the path it took, the confidence it had. Without this, you have no source data to improve against.
Evals
A bench of real cases with known correct answers. New failures get added to the bench so they don't recur. The bench is the ground truth. The team that owns the system writes new evals as part of operating it, not as a one-time setup.
Tuning
A standing process, not a hero project, where the team reviews failures, updates the rubric, and ships. The tuning step might be model fine-tuning, prompt updates, retrieval changes, routing rule adjustments. Whatever the work is, it's scheduled, it's owned, and it produces a delta.
When a compounding system decays, it's almost always because one of those three pieces broke. The logging stopped. The eval bench got stale. The tuning step got skipped because the team got busy. The model is fine. The chain around it isn't.
The cost difference between a one-off and a compounding system isn't the model spend. It's the engineering and ops work to build the chain, and the discipline to keep running it after we leave.
That's why the hand-off phase of every engagement is about the chain, not the model. We can ship a working system in five weeks. The thing that's worth ten times more is whether the team can keep it sharper than we left it.
If they can't, we built the wrong system.