Interviewing
Everyone Says They Double-Check. Ask What the Last One Cost.
Every candidate says they double-check the model, so the claim separates nobody. Ask for the last instance instead: what they checked, what they checked it against, when it happened, and how long it took. Then ask what they decided not to check that same week, and why skipping it was safe. Score whether an outside source appears in the answer, not how confidently it arrives.
The takeThe reflex when the stock answer arrives is to ask it harder, and asking it harder buys a better-worded version of the same sentence. Spend the round on the budget question instead. Someone who says they check everything should be able to say what that cost last week. The habit itself is a way of sorting the few claims that mattered last month from the many that did not, and that sorting is the part of the work that survives a deadline.
Where Olive fits
Open a role and see what the work shows
Olive runs a 40-to-60-minute occupational assignment on the candidate's own clock, with an AI assistant that will complete the whole task if nobody stops it, and a human reviewer writes or edits six findings with the moment in the session behind each one. The candidate is granted the same report, free, on every tier.
Assess your shortlistWhy does pressing on 'I always double-check' return the same sentence?
Because the question is a self-report item, and asked as policy it has one socially correct answer. The candidate knows what that answer is, so pressing harder only rewards whoever words it best. No second or third pass makes the sentence checkable.
The claim turns into evidence only when it acquires a date, an object and a cost.
Measured in one group of job candidates, interview self-presentation came out close to universal, and not all of it was lying.
Julia Levashina and Michael Campion built an interview faking behavior scale across six studies with 1,346 participants. Over 90% of undergraduate job candidates fake during employment interviews, they report, and the behaviors sitting closer to lying run from 28% to 75% depending on which one is counted 1.
Faking on that scale covers tailoring and embellishing as well as inventing, and the people reporting it were undergraduates describing themselves.
But the extra questions do something to the interviewer too. Low-value information dilutes the good information already in hand.
In a 2013 study, 76 undergraduates predicted classmates' semester grade point averages. Predictions made after an unstructured interview correlated .31 with the real figure, while prior cumulative grades alone correlated .65 2.
In a third study from the same paper, 96 of 169 participants ranked an interview in which the interviewee answered at random above having no interview at all 2. Undergraduates predicting a classmate's grades are not managers predicting job performance.
Interviewers make sense of whatever they are handed, including answers with nothing in them, and that mechanism, rather than the size of the effect, is the lesson for a hiring round.
Ask which claim they checked last, and against what
Ask for the claim they checked most recently, and for the thing they checked it against. That pair does most of the work. A habit leaves artifacts, and artifacts have names. A source that got opened. A second run with one number changed. A test that failed. A colleague who already knew the answer.
A policy leaves no artifacts, which is why the policy answer sounds identical coming from a careful person and a careless one.
Then add the clock: how long the check took, and when it happened.
Verification has a duration, and asking for it costs one question. Someone who traced a figure back to a filing can put rough minutes on it. Someone describing a virtue has nothing to time.
From there the general technique takes over. The three rungs that expose whether someone understands the answer they gave work on a verification story as well as on a design decision. Specify which alternative was ruled out, invert the constraint, and ask what would have to be true for the check to have missed.
What counts as an artifact changes by field, and the answer should sound like the work:
- Financial analysis. The figure traced back to the filing, and the line it came from.
- Legal. The citation opened and read, not the citation retrieved.
- Software. The test that reproduced the bug before the fix went in.
- Marketing. The vendor survey behind the market-wide statistic, and how many people it surveyed.
- Operations and support. The runbook step tried against staging before it reached a customer.
The habit has a public name and a public definition. The AI Fluency Framework, written by Rick Dakan of Ringling College of Art and Design and Joseph Feller of University College Cork, files it under Diligence as Deployment Diligence: taking responsibility for verifying and vouching for AI-assisted outputs. The definition names thorough fact-checking, testing for accuracy, and validating claims 3.
No levels and no scoring anchors come with the framework, so writing them is the interviewer's job.
What did they decide not to check?
Ask what went out unchecked last month, and why that was safe. The cost question separates a habit from a slogan, because a habit has a budget and a slogan does not. A strong answer names a class of work: boilerplate that fails visibly, a summary sent to a reader who already holds the document, a draft that a second reader sees before anything ships.
The reason is the part that carries the answer. Cheap to catch downstream, caught by somebody else anyway, or resting on nothing that mattered: each of those is a rule, and a rule is what a person applies when the deadline arrives.
Checking everything is a rule only once it comes with its cost: the hours it took last week.
The sorting rule also survives a change of stack. Someone who can say the model is dependable on formatting and undependable on anything carrying a citation has built a picture of the tool out of use. That picture is why the judgment transfers across tools while the tool habits do not.
A candidate's account of their own budget is a belief about themselves, and beliefs about working with AI have been measured against the work.
One study ran from February to June 2025: 16 experienced open-source developers, 246 real tasks, repositories they had worked on for about five years. They forecast a 24% speedup from AI tools, believed afterwards that they had gained 20%, and were measured 19% slower 4.
A general productivity result would need more than one setting and sixteen developers.
Surveyed in February 2025, 288 Taiwanese K-12 teachers and trainee teachers took both a self-report questionnaire on AI literacy and a knowledge test built on the same framework. The two agreed weakly, at correlations of .07 to .24, and the six profiles the researchers found included teachers who overrated themselves and teachers who underrated themselves 5.
A weak correlation means the two instruments are not measuring the same thing. It does not mean candidates systematically overclaim.
The usable part is the gap between belief and measurement. Neither study measured a claim about checking, so that gap is a reason to ask for the instance behind one.
The skipped case is still a report, but it reports a decision rather than a self-assessment. What went out unchecked last month, and the reason it was safe, is a rule the candidate applied, and a rule can be taken apart in the room.
Score the outside source, not the delivery
Grade one thing: whether anything outside the conversation with the model shows up in the answer. A re-read is not a check, and asking the model whether it is sure is not a check either. The check is a second and independent source of truth: a document, a run, a test, a person, a dataset.
Confidence, fluency and the word always carry no information about whether that source exists.
Write the anchors before the round, because anchored scales have a measured payoff on past-behaviour questions like this one. Levashina, Hartwell, Morgeson and Campion's review reports a 2002 meta-analysis of 19 past-behaviour interview studies. In those studies, anchored rating scales produced higher criterion-related validity, .35 against .26, and slightly higher agreement between raters, .77 against .73 6.
Anchored and unanchored scales were compared across studies rather than inside one interview, so the anchored ones travel with whatever else careful teams already do. The same review reports that anchoring every point of a five-point scale, rather than only the top and the bottom, produced ratings more resilient to disability bias.
So write all three levels, and use them on everyone:
- No. The answer stays at the level of policy, with no instance behind it.
- Partly. An instance with an object and a date, checked against nothing outside the model, a re-read included.
- Yes. An instance with a date, a source outside the model, and what the check settled, whether the work changed or the claim held.
One warning about the error story. Expect one, and do not grade it on how well it is told. The question worth asking is how hard the catch actually was: what made the error visible, and what would have had to be true for it to slip through.
Where the instance question stops working
The instance question rewards memory and articulacy, which are not the thing being hired. A person who verifies well and narrates badly will lose to a person with a well-told story and a weaker habit. Recent graduates, career returners and anyone under a confidentiality agreement may have no dated instance to describe at all, and a round should not read that silence as an absence of the habit.
The fallback is a task, not a better question. Hand over source material with something wrong in it that only checking catches, then grade what got checked and what changed as a result. Hiring for verification rather than production is that exercise with a rubric attached, and it costs an hour instead of five minutes.
A seeded exercise carries its own problems. The packet leaks once enough candidates have seen it. The planted error is easier to find than a real one because somebody chose it to be findable. And an unpaid hour is a real ask at a screening stage.
The double-checking sentence will still arrive early in the next round, and it should: it is the correct answer to the question as asked.
What changes is what gets written beside it. A date, a document, a duration, and one thing that went out unchecked on purpose.
Common questions
What is the single best follow-up to 'I always double-check'?
Ask for the last one. What was the most recent thing checked, what was it checked against, when did that happen, and how long did it take? The four parts matter together: an object with no source is a re-read, and a source with no date is a policy in disguise. A check that ran can be described in specifics. An answer that moves to how quality is generally approached, and stays there, has not yet described one. Ask the same four parts of everyone in the round, so the answers can be compared.
Is it fair to expect a candidate to remember a specific instance?
For work done in the past month, yes, and the question should say so. Recall thins out fast, so anchor it: the last thing checked, this week or last. Anyone whose recent work sits under a confidentiality agreement can give the shape without the content, meaning what kind of claim, what kind of source, how long it took. Someone out of the workforce, or whose last employer banned the tool, has no instance to give at all, and the honest reading there is missing exposure rather than a missing habit.
What does a good answer to 'what did you not check' sound like?
It names a class of work and a reason, and it does not apologise. Formatting and boilerplate that fails visibly. A summary sent to a reader who already holds the document. A draft a second reader sees before anything ships. The reason carries the answer: the failure would be caught downstream, or would be cheap, or the claim was load-bearing for nothing. When a candidate says everything gets checked, the follow-up is how many hours that takes in a normal week.
Should the answer be scored, and how?
Score it against written anchors, decided before the first interview and applied to every candidate. Three levels are enough: policy language and no instance; an instance with an object but no source outside the model; an instance with a date, an outside source, and what the check settled, whether or not the work changed. Across studies of past-behaviour questions, anchored scales showed a measured gain in validity and a smaller one in how closely two interviewers agreed. Anchor every level rather than only the top and the bottom.
Does asking about verification just teach candidates the script?
Partly, which is the argument for asking about instances rather than principles. A script for how somebody verifies is easy to write and word-perfect on the second telling. A script for the last thing checked, when, against what, and what the check settled has to be built out of a real week, and it has to hold up under two follow-ups: what would have had to be true for the check to miss, and what the check cost. Assume the question circulates. Instances still cost more to invent than principles do.
References
- 1. Measuring faking in the employment interview: development and validation of an interview faking behavior scale. ✓ pubmed.ncbi.nlm.nih.gov Supports the claim that self-presentation in interviews is near-universal and mostly mild, so a stock verification answer separates nobody.
- 2. Belief in the unstructured interview: The persistence of an illusion ✓ sjdm.org Supports the claim that low-value interview information dilutes better information, and that interviewers make sense of answers with nothing in them.
- 3. Framework for AI Fluency ✓ ringling.libguides.com Supports the definition of Deployment Diligence, and the statement that the framework publishes no levels or scoring anchors.
- 4. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (arXiv:2507.09089) ✓ arxiv.org Supports the claim that a person's belief about their own AI-assisted work can differ from the measurement in direction, not only in size.
- 5. How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures ✓ arxiv.org Supports the claim that a self-report of AI skill and a knowledge test of it do not measure the same thing, with over- and under-raters both present.
- 6. The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature ✓ doi.org Supports the anchored-rating-scale figures for past-behaviour interviews and the point about anchoring every scale point rather than the ends.
6 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.