What are the tradeoffs between planning and execution in terms of cost, time, and quality?
Test where expensive reasoning creates leverage, and where a grounded plan lets cheaper execution perform just as well.
Starting point
Can cheaper models, given a detailed plan and targeted supervision, match frontier models on well-scoped execution tasks? Yesod should test this rather than assume it.
What to track
- Hold task shape, instructions, tools, verification, and retry policy constant.
- Measure delivery rate, review burden, latency, and total cost—not merely whether the first attempt passes.
- Separate the value of planning from the value of execution.
Working hypothesis
Planning and ambiguity resolution benefit most from frontier models, while routine execution can often move down the cost curve once the work has been grounded.