Free tools · Elle Anderson · andersonco.ukThe AI Feature Readiness Check
Nine questions to ask from the product chair, not the machine learning chair, before an AI feature ships. I have led ML and algorithm-centric products since 2017, and every expensive failure I have watched would have been caught by one of these. Write real answers, not reassurances.
Describe what the customer gets without using the words AI, model or agent.
If it cannot be described, it is a demo, not a product.
What happens, specifically, when the feature gets it wrong? Who pays, and what do they lose?
Charming once, fatal twice. Trust is a metric you design, not one you hope to see.
What does the user see and do immediately after a wrong or low-confidence answer?
The failure state is the product. If it is unspecified, the feature is unfinished.
At what confidence do we show, hedge or stay silent? Who chose those numbers, and on what evidence?
What does the evaluation set contain, who weighted it by customer impact, and when was it last refreshed?
Accuracy on the eval is not accuracy as experienced. The product chair co-owns the eval or the eval measures the wrong thing.
What does one interaction cost today, and what does the curve look like at ten times the volume?
Would this initiative still make sense if the technology were boring? What is the hypothesis, and what evidence would kill it?
Where does the training and grounding data come from, what is it missing, and who does that gap disadvantage?
What measured result, by what date, would retire this feature? Who has the authority to pull it?
Features without kill criteria do not get killed; they get quietly ignored while costing money.