Guide · AI production readiness
10 questions that decide whether a pilot ships
Pilots rarely die on model quality. They die at the review gate, where somebody asks for an audit trail, a rollback path, and a name on the sign-off, and nobody has drafted any of it. Run this list before anyone calls the work ready.
Why the gate is where pilots die
Gartner, in February 2025, expected 60% of AI projects to be abandoned by 2026 where the data underneath was not ready. Gartner also expects more than 40% of AI agent projects to be canceled by the end of 2027. Neither number is about the model. Both are about what sits around it.
The 10 questions
1. Can anyone explain a single output to a person who was not in the room?
If the explanation needs the person who built it, the build has one point of failure and it is a human.
2. Can you roll it back in under a minute?
A rollback nobody has run is a plan. It becomes a control the first time somebody runs it.
3. Do you know what it does when the input is garbage?
Test it with garbage before a customer does.
4. Would you let it run unattended for a week?
If the answer is no, name the reason. That reason is the work.
5. Does every output carry an audit trail a reviewer would accept?
A log an engineer can read is not the same thing. A reviewer has to be able to follow it without help.
6. Is one named human accountable when it goes wrong?
A team is not a name. Ask who gets the call.
7. Has legal, risk, or compliance seen the real thing?
A description is not the thing. They will ask for the thing.
8. Do you know the cost per 1,000 calls at production volume?
Pilot volume hides this. Production does not.
9. Can you tell within an hour that quality has drifted?
Drift is quiet. Nothing breaks, the outputs just get less right.
10. Does it sit in a workflow somebody already uses every day?
A use case nobody performs has nothing to measure and nothing to govern.
How to read the count
What to do with the no answers
Turn each one into an owner and a date. Most of them take days, because the missing piece is a document or a decision and not engineering work. Start with the two that legal and risk will ask about first. Those block the gate no matter how well the model performs.
When this checklist will not help
Anyone with nothing built. This list scores a thing that exists. If the pilot is still a slide, the questions have nothing to grade.
Bring the scored sheet and the launch date somebody has already committed to.