Free tools · Elle Anderson · andersonco.ukThe AI Feature Readiness Check
Nine questions to ask from the product chair, not the machine learning chair, before an AI feature ships. I have led ML and algorithm-centric products since 2017, and every expensive failure I have watched would have been caught by one of these. Write real answers, not reassurances.
Describe what the customer gets without using the words AI, model, or agent.
If it cannot be described, it is a demo, not a product.
What happens, specifically, when the feature gets it wrong? Who pays, and what do they lose?
Charming once, fatal twice. Trust is a metric you design for, not hope for.
What does the user see and do immediately after a wrong or low-confidence answer?
The failure state is the product. If it is unspecified, the feature is unfinished.
At what confidence do we show, hedge, or stay silent? Who chose those numbers, and on what evidence?
What does the evaluation set contain, who weighted it by customer impact, and when was it last refreshed?
Accuracy on the eval is not accuracy as experienced. The product chair co-owns the eval or the eval measures the wrong thing.
What does one interaction cost today, and what does the curve look like at ten times the volume?
Would this initiative still make sense if the technology were boring? What is the hypothesis, and what evidence would kill it?
Where does the training and grounding data come from, what is it missing, and who does that gap disadvantage?
What measured result, by what date, switches this feature off? Who has the authority to pull it?
Features without kill criteria do not get killed, they get quietly ignored while costing money.