Owning the AI Quality Bar vs. Owning the Proof

Deciding whether a feature is good enough to ship has always been product’s call. With AI, answering it takes evidence most product teams have no way to produce, and the standoff that follows is nobody’s fault.

Somewhere in the life of every feature, someone stops and asks whether it is good enough to ship. That question has always belonged to product, and I would argue it is about the most product question there is. Answering it means weighing what a user actually needs against what the thing will cost to build and to keep running, then deciding, with some judgment and a little nerve, where the line falls.

The first article in this series was about a question that comes before that one, whether an item on the feature list is routine work or an open research problem, and how unreliable the usual instincts have become at telling those apart. This one picks up after that has been settled.

Put an AI feature in front of the good-enough question, though, and something breaks. Development asks for the requirements, the way any team would. Then a second request lands with no precedent: to know whether the feature clears the bar, someone has to assemble a ground truth dataset, tens to hundreds of real examples with the correct answers already attached. Without that, there is no reliable way to test a good response from a mediocre-to-incorrect one, and so no way to answer the qualitative question you have always answered. And on many product teams, that job belongs to no one. With no data scientist in the room, progress can stall out.

Product really does own AI quality

It is worth conceding a point to the AI team properly, rather than through gritted teeth, because the literature is genuinely on their side. The field has spent a couple of years converging on the idea that product owns AI quality, and it does not phrase this timidly: Lovelaice calls quality “a product problem wearing an engineering costume,” and Architecture of Proof says an assertion about quality “is not an engineering artifact, it is the PM’s primary instrument of accountability.” I think that is right, and not only for AI. Great has always been the enemy of good enough. Knowing where that line falls is precisely the judgment product exists to make: the point at which the objective is met without overburdening the build, the upkeep, or the cost to run it.

Implementation certainty, and what replaced it

What changed is what used to happen the moment after quality was defined. Once product said what good enough meant, engineering built it, and that it could be built was never really the question. The only open questions were how long it would take and what it would cost, and the thing would go on working for years afterward. I have started calling that implementation certainty, because naming it makes it easier to see how much of the product manager’s job had been resting on its two promises:

1. Whatever product defined, it could be built
2. Once it was built, it would keep working

Put a language model in the middle of the feature and neither promise holds. Whether it works at launch is no longer a given, and whether it keeps working is even less of one, so proving the bar has been met stops being a matter of writing test cases and becomes a matter of probability. So a team can sort the routine features from the research problems and still be stuck.

Engineering could always build it; that certainty is exactly what AI takes away.

Hardware is the right analogy here. A sensor in the field fails in ways that have nothing to do with a manufacturing defect, so reliability is handled statistically and the exceptions get solved before it ships. An AI feature is more like that sensor than like a web form, with one added twist: the component you qualified can change underneath you the week the vendor ships a new model.

The proof of quality has no owner

So where is the team actually stuck? Not on the bar. A product manager can set that alone, with complete authority. Say the feature is an AI router that sends each incoming support ticket to the right team: getting 8 of 10 to the right place is not good enough, and the number to clear is 95%. Nobody is arguing about that.

A number is not proof, though. To know whether the router really clears 95%, someone has to build the ground truth dataset it is measured against, the labeled examples that turn a claim into a score. And that dataset is not a throwaway. The evaluation set that later watches the model for drift in production is built from the same material, so it is the foundation everything downstream rests on.

Assembling it is a discipline of its own, and as a rule it is not a product skill. FutureAGI describes the people who build ground truth working from production traces, customer escalations, and adversarial seeds, and notes that the role “rarely overlaps cleanly with ML engineer or backend engineer responsibilities.” If it does not sit with either of them, it certainly does not sit with a product manager’s Thursday afternoon. That is the gap this series is about: product owns the definition of quality, the request for proof lands on product’s desk, and producing it takes a skill the team was never staffed to have.

How many dimensions the input varies along

The useful question, then, is not how many examples the ground truth dataset needs, which is the one everybody reaches for first. It is how many dimensions the input varies along. A feature that sorts tickets for a single team, in one language, inside one product area, is a contained problem. A few dozen well-chosen examples may be all it needs, and a product person can label that set alone, given a focused sitting or two to do it carefully. However, a feature that pulls the line items out of supplier invoices is a different matter. It spans hundreds of vendor layouts, many currencies, different tax regimes, and every idiosyncratic way a single charge can be worded. That is a coverage problem along every one of those dimensions at once, and problems of that shape are a data science project in their own right.

It is the same request, very often the same sentence in the same ticket, and yet two completely different asks. That is why it is a question worth putting to yourself and the whole team. Whoever ends up building the dataset has to make it big enough to cover the full range of cases the feature will face, and how big that is depends on the answer. The data decides which one you have been handed: a focused sitting or two of labeling, or a project of its own.

Start with the examples that tell you the most

Being handed that harder request is not impossible to fulfill, and it is not a failure on product’s part. The request, however, likely sounds larger than the actual work necessary in the initial stages.

A small set of the examples that separate a good answer from a bad one tells you far more than a sprawling set built for size. Start with the handful that carry the most information:

• the ambiguous cases, where a reasonable person could go either way;
• the ones where a wrong call is expensive; and
• the ones that keep showing up in escalations.

A test set built from easy cases is a trap: everything passes, so a strong score proves nothing.

Start there, and you will not have proven the 95% yet, but you will know whether you are close, which dimensions break things, and where the next batch of examples should come from. That is real progress, and you can make it without anyone new in the room.

Semi-automated labeling, with a scientist in the loop

The other move people reach for is to have the model generate its own ground truth. It can work, but it is not a prompt you fire off once. Done well it is a process, with checks built in at each step to keep the generated labels accurate. Evidently’s guidance on synthetic test data is blunt about what that takes:

• prompting that produces cases like the real ones;
• a metric that measures what you actually care about;
• infrastructure to run and track it; and
• a human checking every generated case.

Enterprise organizations do build ground truth datasets this way and get good results. But it is data-science work: the labeling scheme, the inter-annotator agreement, the sampling that keeps every case represented.

This is, as it happens, the kind of gap we are brought in to fill. The ground truth questions will land in the middle of requirements definition, and most teams need a data scientist in the room to answer them. Building that dataset, self-checks and all, is the work we do at Planorama.

That work has a harder layer underneath it. Working out how many labeled examples it takes before a 95% score means anything, and whether every dimension is represented in the right proportion, is applied statistics, the kind most of us were never trained in.

That is not a mark against product owning the bar, and it is why the standoff that opened this piece was never really a fight. Both sides are right: product owns the bar, and clearing it now takes work no one on the team was hired to do. The way out is to name that work and put the right person on it. Do that, and what felt like a standoff turns back into a plan.

This is the second of four articles on the work an AI feature demands before anyone can write requirements for it. The next one goes underneath both of these questions, to why the specification could not have been written first, no matter who was holding the pen.