The efficiency question starts before the answer
When an AI system answers a business question, some of its work is finding the right records. That can mean locating a contact, matching a company, checking a job and reconciling an invoice. The access architecture is part of the cost of the answer. Holiday's latest internal study compares two ways of doing that retrieval work.
Keep the model and records fixed
The study describes a controlled comparison in which the model and underlying records stayed the same while the access architecture changed. One workflow navigated separate systems through MCP tools. The other used the Holiday Environment. This comparison addresses retrieval design; it does not establish that one model is universally better than another.
Define what a complete answer requires
The task required connecting correspondence to the correct business, checking relevant history, inspecting work and retrieving related accounting records. A useful evaluation must score the information required to answer the question, not merely whether the response sounds convincing. The question, expected facts and scoring rubric should be recorded before comparing runs.
Measure tokens, calls and quality together
Token use alone is not enough. Record the agent-facing calls and whether each answer includes the required information. For cost claims, distinguish token counts from actual charges, because input, output and cached tokens may have different prices. Timing also needs its own measurement. A token reduction does not prove an identical percentage reduction in latency or total operating cost.
State the limits of an internal benchmark
This is Holiday's internal study, not an independently audited customer result. A bounded task can show how an architecture behaves under those conditions. It cannot by itself establish a saving for every company, model, integration or workload. The next useful step is to repeat the comparison across more tasks, with clearly recorded configurations and quality checks.
Use the study to design your own test
Choose a business question that matters, use the same records and permissions for both approaches, and compare complete outcomes. Include a case with conflicting information and one with missing information. Treat a system that knows when to stop as more useful than one that returns a confident but incomplete answer.
What the comparison is for
The Holiday Environment connects business knowledge in a per-company environment. The study asks whether that structure changes how much retrieval work the AI must do. The practical lesson is to evaluate how information reaches the model as well as which model you choose. The full study is linked below so the comparison can be inspected rather than reduced to a headline.