The short answer
Errors, latency, cost and reviewability all matter. Compare models on a fixed task rather than treating a leaderboard as a business decision.
Try it in three steps
- Prepare a small set of normal, incomplete and edge-case tasks.
- Keep inputs and prompts fixed and save raw outputs.
- Compare total cost including human correction, not just request price.
A practice prompt
Before you start
Use three fixed tasks for a comparison table, leaving unmeasured columns blank.
Practice material (fictional)
Tasks: extract pencils, 3 boxes; extract erasers, quantity unspecified; answer refund timing when the policy is silent.Replace placeholders in the prompt with this material, or append the material if there are no placeholders. Reading time excludes doing the exercise.
Design a comparison table for accuracy, omissions, latency, correction time and cost. Leave unknown values blank.What to look for (not a recorded test result)
Check 3 boxes, unknown quantity and unknown timing, then measure latency and correction time. No untested model can be declared cheaper overall.
If it goes wrong
If tools or source materials differ, do not attribute every difference to the model.
Check your result
You can choose a sufficient option for tested tasks and name untested scenarios.
References and scope
The links below support the concepts or product navigation. The steps and exercises are our teaching suggestions, not official certification or measured customer outcomes. Product interfaces and availability may change. References checked on 2026-10-17.