The demo always goes well. Someone opens a chat screen, types a rehearsed question, the agent answers in two seconds with a correct and elegant response, and the room approves the budget. Three months later that same agent is promising a customer a discount that does not exist, nobody knows why it said so, and the case cannot be reproduced because nothing was logged.
The short answer: the demo showed two layers, the interface and the agent. The system has thirteen. The eleven you did not see are not engineering decoration: they are the ones that answer what happens when the model is wrong, who authorizes what it cannot decide alone, and how you know it works better today than last month. A project does not fail because of the model. It fails because of the layers nobody built.
Why do projects get canceled when the pilot worked?
Because the pilot is judged on selected cases and the system is judged on reality.
Three different sources, measuring different things, point the same way. Gartner projects that over 40% of agentic AI projects will be canceled before 2028, attributing it to escalating costs, unclear business value and inadequate risk controls [1]. IDC estimates that 88% of AI proofs of concept never reach wide deployment [2]. And MIT's study of 300 deployments found that 95% of generative AI pilots produced no measurable return [4].
Data
| Measurement | Percentage |
|---|---|
| Generative AI pilots with no measurable return | 95% |
| Proofs of concept that never reach wide deployment | 88% |
| Agentic AI projects to be canceled before 2028 | 40% |
None of the causes Gartner lists is a model problem. Cost, value and controls are system problems, and all three live in the layers the demo did not show.
What are the 13 layers?
Thirteen, grouped into five functions. A demo usually has the first two.
What the user touches
| Layer | The question it answers | What breaks without it |
|---|---|---|
| Interface | Where does the conversation happen, and is the customer already there? | The system works but nobody uses it, because it asks the user to go somewhere new |
What decides
| Layer | The question it answers | What breaks without it |
|---|---|---|
| Agent orchestration | Who does each step, and in what order? | One agent attempts everything, and errors compound step by step |
| Business rules | What is true in this company and not negotiable? | The model improvises discount policies, deadlines and warranties nobody approved |
| Model routing | Which task deserves which model? | You pay large-model prices for small-model work, and the production bill arrives multiplied |
What it knows
| Layer | The question it answers | What breaks without it |
|---|---|---|
| Context and memory | What does it remember about this customer and this conversation? | The customer repeats their story at every interaction |
| Data foundation | Where do its claims come from, and how fresh are they? | It answers with total confidence from an eight-month-old catalogue |
| Connected tools | What can it do besides write? | It describes the action instead of performing it, and somebody does it by hand |
What keeps it honest
| Layer | The question it answers | What breaks without it |
|---|---|---|
| Approval checkpoints | What can it not decide alone? | An irreversible action executes with nobody having authorized it |
| Testing and evals | How do I know the answer is good? | Quality is judged by anecdote, and an apparent improvement hides a regression |
| Security and access | Who can see what? | The agent answers one customer using another customer's data |
| Tracing and logs | What exactly happened in this case? | An incident cannot be reproduced, so it cannot be fixed |
What improves it
| Layer | The question it answers | What breaks without it |
|---|---|---|
| Learning loops | How does the error get back into the system? | The same failure repeats for months because nobody feeds it back into the design |
| Constant refinement | Who maintains it when the models change? | The system freezes at its launch-day version and ages on its own |
Which layer does almost everyone skip?
Evaluation. And one figure shows it with uncomfortable precision.
LangChain's State of Agent Engineering report found that 89% of teams with agents already have observability instrumented, but only 52% run offline evals and just 37% run online evals [3].
Data
| Practice | Teams |
|---|---|
| Have observability instrumented | 89% |
| Run offline evals | 52% |
| Run online evals | 37% |
The distance between 89 and 37 is a mentality LangChain names well: ship and watch. Having the trace without the eval is having the camera footage with nobody reviewing it. It helps you investigate after the incident, not prevent it.
On the executive side the number matches: only half of executives regularly verify the quality of what AI produces [5].
Is ambition moving faster than capability?
Considerably faster, and that distance is where projects get canceled.
Data
| Today | Planned within two years | Delta | |
|---|---|---|---|
| Organizations with agents deployed | 17% | 60% | 43% |
Going from 17 to 60 in two years is not a model problem. Models are available to anyone with a credit card. It is a problem of the eleven layers.
Does a small project need all thirteen?
No, and demanding them from day one is an expensive way to never ship.
Here is the honest limit of this framework: most projects should not build thirteen layers before having a user. What matters is telling apart what is cheap to add later and what is not.
Mandatory from day one, because adding it later means rebuilding: data foundation, security and access, and tracing. All three define the shape of the system. Retrofitting access control onto a system that already stores everything together is a project, not a tweak.
Added when usage demands it: orchestration, model routing, memory, learning loops. Building them before you know what production looks like is designing for an imagined problem.
Never optional, at any size: approval checkpoints for irreversible actions. A small agent that can issue a credit note needs the same control as a large one.
How we apply this at MasterDragon
We start with the layer that hurts most to lack, not the one that demos best.
In practice, the first conversation on a project is not about the model. It is about which actions would be irreversible, which data must never cross between customers, and how we will know the system got worse before an angry customer tells us. With that settled, the agent is the fast part.
It also means we ship with tracing on and with a set of evaluation cases from the first deployment, even if it is twenty hand-written cases. Twenty cases that run on every change are worth more than an observability dashboard nobody opens.
If you are evaluating an AI proposal and want to know how many of these layers it includes, talk to our engineers before signing. You can start with how we build your software and review our portfolio of shipped AI-native products. How much autonomy to give each agent we develop in engineered agency, how to design the system so it can be changed in parts in composable enterprise architecture, and why using the tool is not enough in adoption is not adaptation.
References
- Gartner. (2025, June 25). Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
- International Data Corporation. (2025). Estimate on AI proofs of concept failing to reach wide deployment, widely cited in trade press.
- LangChain. (2026). State of Agent Engineering. https://www.langchain.com/state-of-agent-engineering
- Project NANDA, Massachusetts Institute of Technology. (2025). The GenAI Divide: State of AI in Business 2025.
- Cantrell, S., Domergue, C., Dake, A., Murphy, J., Sundholm, T., & Gustafson, M. (2026, July 9). AI adoption to adaptation. Deloitte Insights. https://www.deloitte.com/us/en/insights/topics/talent/ai-adoption-to-ai-adaptation.html