Description Is Trainable; Discernment Is What You Hire For

Which AI competency is actually scarce?

Discernment: judging what came back, including noticing that a confident, well-formatted answer is wrong. It is scarce because it is not really an AI skill. It runs on knowing a domain well enough that a wrong number feels wrong before anyone can articulate why, and that knowledge accumulates over quarters of doing the work rather than over an onboarding week.

The named version of the four competencies is delegation, description, discernment and diligence, and what each one looks like in real work is worth reading before weighting them, because the framework itself is deliberately neutral about which matters most. It was written to teach. A requisition has to choose.

The cost of missing discernment has been measured once with unusual clarity. In a field experiment with 758 consultants at a large firm, on a single task deliberately placed outside the model's capability, the group using GPT-4 was 19 percentage points less likely to reach the correct answer: 84.5% of the control group got it right, against 60% and 70% in the two AI conditions ¹. One task, one sample, a 2023 model, and the capability line moves with every release. What travels is the shape of the failure. Those consultants did not lack a fact. They could not tell which side of the line the task was sitting on, and the model gave them no clue, because a model outside its competence sounds exactly like a model inside it.

Why is description trainable and discernment not?

Because one improves by repetition and the other by accumulation. Specifying a task is a habit: name the constraint, name the audience, hand over the source material, say what a wrong answer would look like. A new hire picks it up from watching two colleagues do it for a fortnight. Discernment cannot be practised in the abstract, because what looks wrong depends entirely on what a right answer usually looks like in this field.

The productivity literature backs the first half of that, if it is read as a claim about which part of the work the tool substitutes for rather than as a claim about output. In a staggered rollout of a conversational assistant to 5,179 customer support agents at one software firm, issues resolved per hour rose 14% on average, with a 34% improvement for novice and low-skilled agents and close to nothing for experienced ones ². Pooling three company-run randomized trials across 4,867 developers, an AI coding assistant raised completed tasks by 26.08%, with a standard error of 10.3%, and less experienced staff both adopted it more and gained more ³.

The standard error on that second figure is wide enough that the range, not the point estimate, is the honest reading. Both studies also measure volume on tasks with correct answers rather than judgment on open work, in different firms and different eras. Taken together they still show one consistent thing: the tool closes the gap fastest exactly where the work is production, and it closes nothing where the work is knowing the produced thing is wrong. That last inference is a reading of the pattern, not a finding either paper reports.

The weighting follows from that pattern. If a tool levels up the specification-and-production half of the job for junior staff within weeks, paying a premium at hire for that half is paying for something you were about to get anyway.

Ask for the wrong answer they caught

One question decides more of this round than the rest of the loop. Hand the candidate material out of your own work with something wrong in it, then ask what they would do with it. Prompt quality, tool names and speed all move around that question. What you are listening for is whether the error surfaced at all, and what got checked in order to surface it.

Two ways to run it, in ascending cost:

1. The recall question, ten minutes. "Show me something an assistant gave you that you did not use, and how you knew." It works in a phone screen and it returns an account rather than an act. The follow-up is the whole question: what did you open, run, recompute or call to find out. 2. The seeded exercise, twenty minutes plus an afternoon to build. Real material from the role with one confidently wrong thing in it, an assistant available, and a rubric written before the error was planted. How to plant an error a candidate's field would catch is the design problem worked through.

What counts as catching it is naming the thing checked against. "It felt off" is taste, which is weak evidence and unevenly distributed by how confidently somebody talks. Opening the filing, re-running the query, reading the payer policy or calling the customer is verification, and it is the same move in every function even though its object changes.

Diligence belongs next to discernment for the same reason: it is a disposition rather than a technique. The question there is whether the person withdraws a claim they cannot stand behind, which nobody learns during onboarding either. Hiring for verification rather than production is the version of this argument aimed at the job description.

Where does a weighted bar go wrong?

Three ways, and the first is the one nobody notices. Weighting discernment means testing your domain, so the exercise has to be built out of your material or it measures nothing at all. The second is dropping description to zero, which is not what a floor means. The third is grading the catch rather than the check behind it.

• It is domain-bound, so it does not travel. A candidate strong in revenue-cycle work will miss a planted error in a marketing brief, and that is the instrument working correctly rather than the candidate failing. Never carry a discernment result across functions, and never let one panel's exercise become the company's standard exercise.
• A floor is still a bar. Somebody who cannot brief a model at all will burn a week of a senior's time in month one. Test description as a pass or a fail on one short task, not as a scale that a strong specifier can use to outrank a strong checker.
• Grade the check, not the catch. A candidate who missed the planted error but described exactly what they would have opened to verify the claim has shown more than one who spotted it by luck and cannot say how. Write that into the rubric before the first interview, because it is the distinction panels lose first.
• And the ceiling on all of it. An interview gets you a description of a check, never a check. That gap is why the exercise exists, and it is also why the word on the posting matters less than the behavior underneath it: what fluency means and who gets to set the bar.

See how it works >