What Does Being Good at Using AI Look Like in an Interview?

What are the three behaviors worth watching for?

Framing, evidence, and verification. A candidate who is good with AI states the problem in their own terms before asking for output, demands a source for the one claim the answer rests on, and checks something against the world outside the chat. Everything else (tool names, prompt syntax, how fast the draft arrived) is a proxy for those three, and a weak one.

• Framing before generating. The first move goes after understanding rather than output. It names what is being decided and what would make an answer wrong, in the candidate's own words. The failure shape is asking the assistant for the deliverable in the first message and letting the framing arrive with it.
• Evidence for the claim that matters. Not "add citations": a source demanded for one specific load-bearing claim, and actually opened. The failure shape is confident assertions flowing into the deliverable as facts, with nothing asked for behind them.
• A test against something outside the chat. A number recomputed, a script run, a page opened, a person called. The failure shape is treating fluent description as a result, which is the single most expensive habit a new hire can bring.

Those three are not a taste judgment. A survey of 319 knowledge workers, describing 936 real tasks they had done with generative AI, found the thinking does not disappear when a model arrives. It moves, from gathering information toward verifying it, integrating a partial response, and stewarding a task the model is doing part of ¹. What is left after the drafting is automated is exactly what an interview should be asking about.

If you want the same idea as a scored instrument rather than a question list, the four Ds rubric is the closest widely-used framing, and it maps onto these three moves cleanly enough to borrow.

Why does verification look different in every field?

Because the thing you check against belongs to the job, not to the chat. A financial analyst ties a number back to the filing. An engineer runs the code and reads the failure. A marketer opens the campaign data the model summarized. A paralegal pulls the case. The move is the same; its object is the field's, which is why one generic definition of AI skill cannot be assessed.

The stakes scale with the domain. Domain-specific legal research tools (not general chatbots, but products sold to lawyers on the promise of grounded answers) hallucinated between 17% and 33% of the time in a 2024 Stanford evaluation ³. Nothing in the interface distinguished a usable memo from a sanctionable one. Somebody opening the cited case did.

So write your question against your own material. "How do you check AI output" gets you a philosophy; "walk me through the last time you caught something wrong in a model's answer, and what you opened to catch it" gets you the field-specific act or nothing. If your team spans several functions, the check differs per function, which is a design constraint on any competency framework you write, not a detail to smooth over.

Ask what they threw away, not what they use

The most informative question in the round is about a rejection. Ask for one thing an assistant produced that the candidate did not use, then ask how they knew. A candidate who opened the source, re-ran the number or called the person is describing verification. A candidate who says it sounded off is describing taste, which is a much weaker piece of evidence.

Four questions, in this order, do most of the work:

1. "Take me through the last time you used it. What were you deciding?" You are listening for a task with a decision attached, not a tool name. A four-product inventory with no task behind it is where thin claims stop, and the pattern is common enough to have its own failure mode. 2. "What did you ask it before you asked it for the output?" Framing shows up here or nowhere. Silence is an answer. 3. "Show me something it gave you that you didn't use. How did you know?" The rejection question. The follow-up is the whole question; the first half is just setup. 4. "What did you deliberately keep and do yourself?" Someone who judged the model was the wrong instrument for a step, and says why, has demonstrated the thing you are hiring for.

One caution about who scores well here. In that same survey, higher confidence in the tool was associated with *less* critical thinking, while higher confidence in their own ability was associated with more ¹. The most fluent AI talker in your pipeline is not automatically the one doing the checking, and the two are easy to confuse in a 45-minute round. Push past the first answer every time. The follow-up is what exposes understanding, because a rehearsed account survives one question and rarely survives three.

What doesn't count as being good at AI?

Volume, speed, tool inventory and prompt syntax. None of them survives contact with the job. Sixteen experienced open-source developers in a 2025 randomized trial took 19% longer to complete real issues in their own repositories when AI tools were allowed, and still believed afterwards that the tools had sped them up by 20% ². If practitioners misread their own throughput that badly, a candidate's account of their gains is not evidence.

• Volume of use is not a virtue. Heavy use with no refusals is worse evidence than occasional use with a clear account of what got thrown out.
• Speed is not judgment. The candidate who produced the deliverable fastest may simply have skipped the checking, and in a 45-minute conversation those two look identical.
• Certificates and tool lists price in nothing. They say a course was completed, not that a wrong answer was ever caught.
• Polish is a distractor. Every candidate's answers are more fluent than they were two years ago, which is why the same well-shaped STAR answers keep arriving and why fluency has stopped carrying signal.

The useful inversion: stop asking what the candidate can produce with a model and start asking what they refuse from one. Production is the part that got cheap. Hiring for verification rather than production is the same argument applied to the job description, and it changes what the interview is for.

Where does an interview stop?

At description. An interview can capture a candidate telling you how they checked a confident claim last quarter; it cannot capture them checking one. What you get is a reconstruction, filtered through how well someone narrates their own work under time pressure. Narration is a separate skill, unevenly distributed, and one plenty of strong practitioners have less of than the people they outperform.

That ceiling is worth stating plainly, because the fix is cheap and most teams skip it. Write the three moves down as a rubric before the first interview, ask every candidate the same four questions in the same order, and record what the answer contained rather than how it felt. Even an informal or casual interview counts as a selection procedure once it decides who advances ⁵, and the federal guidance on those procedures is to administer them the same way for every candidate ⁴.

Past that, the only instrument that closes the gap is one where the work happens in front of you: a short task on occupational material, with an assistant available and a rubric written beforehand. That is a work sample rather than an interview, and it costs an afternoon to build. Replacing an online assessment that AI has already solved usually funds it. Multiple-choice AI literacy tests, code-collaboration graders, unwatched take-homes and live AI-open rounds are all real approaches with different trade-offs; what none of them can do is tell you what happened, moment by moment, unless the session was recorded and read by a person.

See how it works >