After the launch of the Framer Agent, we started digging into how we could make the Agent radically cheaper by tracing the compounding cost through real sessions. Three sources accounted for most of the waste:
Cache misses that made the model process old context again at the full input-token rate
Tool steps spent managing the agent rather than improving the result
Diagnostic payloads that were much larger than the model needed
Fixing them made a representative trajectory dramatically leaner. In head-to-head comparisons we observed up to 67% fewer steps, up to 64% fewer fresh input tokens, and up to 45% less tool-result data, with no regression on CanvasBench.
Before & after
This comparison shows the same task running through the old and new harnesses where a near identical result was achieved. Lower is better for every metric shown.
Metric | Before | After | Change |
|---|---|---|---|
Steps | 36 | 12 | -66.7% |
Tool Payload Bytes | 249.0 KiB | 137.6 KiB | −44.7% |
Cached Replay Cost | $1.969 | $0.548 | −72.2% |
Fresh Input Tokens | 341,690 | 124,378 | −63.6% |
Eliminating cache misses
Why token caching matters
An agent sends the model a growing conversation on every step: its instructions, the user’s request, previous model responses, tool calls, and tool results. Most of that input is identical to the request before it.
Token caching allows the model provider to reuse work already performed for an unchanged prefix of that conversation. Those reused input tokens are substantially cheaper than tokens the provider must process from scratch.
A cache miss occurs when that reusable prefix cannot be found or no longer exactly matches. The provider then processes those tokens again as fresh input. In a long agent trajectory, even one miss can be expensive: later steps may replay hundreds of thousands of tokens, so a small break in cache continuity can dominate the cost of the run. As a side-effect, cache-misses also make the next result slower.
Our target was therefore simple: after the first request, every subsequent request should reuse the longest possible prefix.
Eliminating full cache misses
When we began inspecting production traces, we found far more full cache misses than expected. We expected zero, but we were seeing traces that had 2 or 3 such events.
We identified hosting issues that could cause a cache miss when a request was rerouted to a healthier provider. This could occur multiple times in a tool loop.
We also identified a persistent cache miss which we isolated to the response to the first image added to a session that had previously contained only text.
Working closely with our model provider/routing partners, we resolved both of these issues. To highlight the significance of these improvements, eliminating just a single cache-miss from the Before & After table trace saved $0.6883.
A cache-hostile compaction strategy
After eliminating the full misses, our own architecture was still invalidating roughly 10–20% of otherwise reusable context multiple times in a trajectory.
The old harness encouraged the model to make several large batches of changes and then request the complete diagnostics in a batch. Those diagnostic results were large, so when the model requested a new review, we removed the previous result from its trajectory to keep the context smaller.
That looked like an input-token optimization, but it worked against prefix caching. Removing an earlier message changed the conversation at that point, so everything after it stopped matching the cached prefix. We saved the old diagnostic tokens but paid the full input rate to process the intervening context again. Once we measured the tradeoff, the answer was clear: preserving a stable prefix was worth more than redacting the old result. We removed that compaction behavior and brought avoidable cache misses down to zero.
Removing unnecessary tool steps
During development, Sonnet 4.6 was one of our target models. In our harness it sometimes stayed in reasoning mode when the next action was critically ambiguous, burning tokens without getting anything done.
To help, we built an exciting checklist tool. It let the model declare its planned work and the dependencies between tasks. The runtime could use that structure to guide the agent programmatically toward its next action, making the tool much more powerful than simple interface management step, actually steering the model when the next step might have otherwise been ambiguous. The tool solved the problem it was designed for, and we initially exposed it to every model.
Launch data showed that this was the wrong default:
The stronger models did not need the additional guidance
Maintaining the checklist introduced steps whose only effect was updating the checklist
In an agent loop, every tool step creates another model request, and every request replays the trajectory so far, so a bookkeeping step costs more than its small output suggests.
We removed checklist management only from state-of-the-art models, and observed a large reduction in replayed token costs. In 24 step trace building a webpage from scratch, we eliminated 7 steps, while CanvasBench performance remained unchanged.
Shrinking diagnostic payloads
The final source of waste was the diagnostic review itself.
Our diagnostics included a structured canvas diff covering all changes the agent had made. This was useful: the agent could see unintended side effects and correct them. But the review arrived as one large payload after a batch of changes. Once we stopped removing old diagnostics to preserve the cache, those payloads stayed in the trajectory and were replayed on every later step.
We redesigned the feedback loop rather than simply truncating its output.
Each bulk update now returns diagnostics for that update immediately, eliminating a step right away. We made the diagnostic format denser and stopped serialising full node diffs by default.
Moving detail behind a tool can backfire if vague messages force the agent to make several follow-up calls. To prevent that, we audited the full set of diagnostic messages and made each one precise enough to identify the problem, its location, and the likely next action without the accompanying diff. The agent now fetches node data only when the extra detail is actually necessary.
The result was between a 30-45% reduction in tool-result payload size, without increasing tool use.
What we learned
Together, all of these changes brought the average cost of a Framer Agent session with any model down by 40-48%, with model-specific fixes taking that number even higher.
Even with such a big improvement there are still many more opportunities for us to chase! I’m looking forward to sharing more updates in the future that bring these numbers down more.
