Interviewing
The Refusals Worth Trusting Have Moved at Least Once
A strong answer to what a candidate decided not to use AI for lands on one task, not a category. Keeping the recommendation while the model drafts the summary is a boundary. Refusing anything confidential usually reads back a policy. A strong answer also names the consequence and who absorbs it. And the line has moved at least once, so the candidate can say what they used to keep and now delegate, or the reverse, and what changed their mind.
The takePanels reward the largest refusal, and size carries almost no information. A candidate who will not put anything client-facing near a model sounds careful and has usually told you which policy they read. The answer worth trusting admits a line was drawn in the wrong place once. A boundary tested against a result is the only kind that teaches anybody anything. And the weakest answer in the set comes from the candidate who has found nothing worth keeping at all.
Where Olive fits
Open a role and see what the work shows
An interview can capture a candidate describing the boundary they keep, but not holding one while a capable assistant offers to cross it. Olive runs that as work instead: a role-grounded assignment with an assistant that will do the whole thing if nobody stops it, and a human reviewer who writes or edits the findings on what the candidate kept and why.
Assess your shortlistWhat does a real boundary sound like out loud?
A real boundary lands on a task, and a weak one lands on a category. Categories are what a policy hands out: nothing confidential, nothing client-facing, nothing that goes to a regulator. Tasks are what the job is made of, and a candidate who has done the job can point at the one they kept and say what it produces that the drafting step does not.
O*NET, the US Department of Labor's occupational database, lists eleven task statements for management analysts. The first is to gather and organize information on problems or procedures. The third is to analyze data gathered and develop solutions or alternative methods of proceeding 1.
The two statements sit two lines apart, and nothing guarantees they behave alike under an assistant.
A candidate who hands over the first and keeps the third has drawn a line you can inspect. A candidate who says they keep the analysis has named a word. Ask which analysis, on what input, producing what, and the difference shows up inside a sentence.
Pull the list before the round. Read the task statements for the occupation you are hiring into, mark the three that carry the most consequence, and ask which of the three the candidate would not delegate today. Your own team has already answered a version of that question, if anyone there has ever decided which tasks are worth keeping out of the model.
The candidate's answer is most useful where it disagrees with yours.
Why does the reason outrank the size of the refusal?
Because the reason is the part that travels to a task nobody has met yet. A refusal with a consequence attached tells you what the candidate is protecting and who absorbs the damage when the protection fails. A refusal with nothing under it is a rule that stops working the moment the role, the tool or the client changes, and the candidate has no way to notice.
A pre-registered field experiment at Harvard Business School in 2023 put something close to that question to 758 Boston Consulting Group consultants, split across two experiments. The 385 assigned to the first worked eighteen realistic consulting tasks picked to sit inside the model's capability, with GPT-4 given to two of the three randomly assigned groups. Those with the model completed 12.2% more of the tasks than the control group 2.
The other 373 got a second experiment built on a single task chosen to sit outside that capability. On that one, consultants using GPT-4 were 19 percentage points less likely to reach the correct answer, and the group given a prompt-engineering overview did worse than the group given none 2.
One task, one firm, one sitting, and a mid-2023 model. The shape of the failure outlasts those limits: many consultants handed GPT-4 could not tell that this task sat past the line of what the model could do.
A boundary is a guess about where that line runs, and only the reason under it says how good the guess is.
Rick Dakan of Ringling College of Art and Design and Joseph Feller of University College Cork wrote the AI Fluency Framework, which lists Delegation alongside Description, Discernment and Diligence. They define Delegation as creative vision plus the selection of the right AI tools and techniques to realize that vision 3.
Levels, scores and thresholds are the parts the framework does not publish, so what each of the four D's looks like at work gives you a vocabulary rather than a bar.
The question is aimed at delegation, and a good answer draws on discernment too. Deciding what not to hand over is a judgment about what the tool produces, and among the four competencies discernment is what you hire for.
Ask when the line last moved, and what moved it
Put a date in the question, because a boundary that has never moved is a rule nobody has tested against a result. What did you stop doing yourself in the last year, and what made you stop. What did you take back from the model, and what did it cost before you took it back.
Both directions count.
Movement is the cheapest available evidence that a test happened. Somebody handed a task over, watched what came back, and either kept the arrangement or reversed it.
Failed answers look alike. The line was set during onboarding, nobody has revisited it since, and the candidate describes it in the words of the document it came from. A candidate who has used these tools daily for two years and whose line has never moved in either direction is telling you the tool never surprised them once.
The whole answer is the candidate's account of their own judgment, which is the limit worth holding on to. In a February 2025 survey, 288 Taiwanese teachers took both a self-rating and a knowledge test of AI literacy built on the same four-part framework. Correlations between what they reported and what the test measured ran from 0.07 to 0.24 4.
Teachers rather than job candidates, and a survey rather than a hiring round. The study also found teachers who underrated themselves, so the gap between the two measures runs both ways.
Treat a boundary answer as a lead to test, the same way you would treat a candidate who says they always check the output.
The stronger move is to stop asking and watch: put a task in front of them, let the assistant overreach, and see where they stop. Then somebody has to decide before the session when you are allowed to intervene, which is a problem of its own.
Score the refusal on three things, not on how careful it sounded
Write three columns before the round: the task named, the cost named, and whether the line has moved. Fill them from what the candidate said rather than from your read of them. Ask every candidate the same question in the same words, and have two interviewers fill the columns separately. A boundary answer scored on impression is a conversation you enjoyed.
- The task. Does the boundary land on something in the role's own task list, or on a category a policy would have supplied.
- The cost. What goes wrong when the line is crossed, and who absorbs it: the client, the reader, the next person on call, the candidate.
- The movement. When the line last changed, in which direction, and what evidence changed it.
An unscored conversation can leave you worse off than no conversation. In a controlled test, 76 undergraduates predicted classmates' semester grade averages. Predictions made after an unstructured interview correlated .31 with the actual result, against .65 for predictions from prior grades alone 5.
In a later study in the same paper, 96 of 169 participants ranked an interview in which the person answered at random above conducting no interview at all 5.
Undergraduates predicting a classmate's grades are not managers predicting job performance.
The mechanism is the same: a conversation that tells you little dilutes what you already know, and an interviewer will make sense of almost anything, including answers generated at random.
The question does not work in every building. A candidate working under a contract that bars third-party processing has no line to set and none to move, and asking them to produce one rewards whoever happened to work somewhere lax.
So run the question with the task list open in front of you. Once the candidate names the statement they keep today, ask which one they would have kept a year ago.
The second half is the part nobody rehearses.
Common questions
What does a weak answer about not using AI sound like?
A category with no cost and no history behind it. Nothing confidential, nothing client-facing, nothing that goes to a regulator: each names a class of material rather than a task, and each is the kind of line a company policy supplies. Weak answers also stay still. If the candidate cannot say when the line was drawn, who drew it, or what would move it, the boundary was probably inherited and never tested. Follow up once before writing the answer off, because a nervous candidate often has the specific version sitting underneath the general one.
Should a candidate who refuses to use AI for anything be rejected?
Not on the refusal alone, and not on the size of it. Ask what the refusal costs them: which tasks take longer, what they gave up to keep the line where it is, and what evidence would move it. A candidate who can price the refusal has thought about it. A candidate who cannot has adopted it. The job matters too: a role where the whole team drafts inside a model every day is a poor fit for a candidate who will not read a model's output.
How do you tell an inherited policy from a boundary the candidate set?
Ask who wrote it. A policy has an author, a date and a scope the candidate can describe, and a candidate repeating one will usually say so plainly when asked. A boundary the candidate set comes with a story: a specific task, something that went wrong or nearly did, and a decision made afterwards. Both are acceptable answers. Only one tells you what the person does when the policy runs out, which is the situation the role will put them in.
Does the candidate need a work example to answer this well?
No, but the answer needs a task and a consequence, and a work example is the easiest place to find both. A candidate under a confidentiality agreement may be able to describe the shape without the client, within what the agreement allows: which step they kept, what the failure would have cost, who would have absorbed it. Coursework, volunteer work and personal projects all carry real boundaries. What does not work is a general statement about being careful, because every candidate says that and it separates nobody.
What if the candidate's boundary is wider than the one the team keeps?
Treat it as information about the team as well as about the candidate. A person who keeps more than your standard practice has usually paid for the difference somewhere, and the reason is worth hearing before the panel calls it slow. If the reason names a consequence your team also carries, the disagreement is about where the line sits rather than about judgment. Write down what they said, put it beside your own practice, and decide which version you can defend to the person who owns the risk.
References
- 1. Management Analysts (13-1111.00) ✓ onetonline.org Primary source fetched 2026-09-01, page marked Updated 2026: the Tasks section is headed '5 of All 11 displayed' and the full list carries eleven task statements, the first 'Gather and organize information on problems or procedures.' and the third 'Analyze data gathered and develop solutions or alternative methods of proceeding.' Supports the article's claim that a candidate's boundary can be located on a named task from a public occupational task list.
- 2. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality (Working Paper 24-013) ✓ mitsloan.mit.edu Live read of the PDF 2026-09-01. Supports both halves of the article's frontier claim, with the cohort as the paper states it: the pre-registered study allocated 758 Boston Consulting Group consultants to one of two distinct experiments with no overlap, 385 to the inside-the-frontier experiment and 373 to the outside-the-frontier one, each randomly assigning participants to no AI, to GPT-4, or to GPT-4 with a prompt-engineering overview. On the 18 inside-frontier tasks the AI groups completed 12.2% more tasks than the control group; on the single outside-frontier task they were 19 percentage points less likely to produce correct solutions (control 84.5% correct against 60% and 70% in the two AI conditions), the overview group down 24 points against the no-overview group's 13.
- 3. Framework for AI Fluency ✓ ringling.libguides.com Live read 2026-09-01 of Framework for AI Fluency V 1.1: the four core competencies are Delegation, Description, Discernment and Diligence, and Delegation is defined on the page as 'Creative vision and selection of the right AI tools and techniques to realize that vision.' The page publishes subcategories but no levels, scores or proficiency thresholds. Authors Rick Dakan (Ringling College of Art and Design) and Joseph Feller (Cork University Business School, University College Cork).
- 4. How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures ✓ arxiv.org Live read of the arXiv PDF 2026-09-01. Supports the article's caution that a boundary answer is self-report: data collection ran in February 2025 among Taiwanese pre-service and in-service K-12 teachers, the analytic sample was 288 who completed both the self-report and the objective instrument on the shared Concept / Use / Evaluate / Ethics framework, correlations between the two sets of factors were 'consistently weak, ranging from r = 0.07 to r = 0.24', and the six latent profiles include underestimation as well as overestimation.
- 5. Belief in the unstructured interview: The persistence of an illusion ✓ sjdm.org Live read 2026-09-01. Supports the article's instruction to score the answer rather than react to it: in Study 1, 76 Carnegie Mellon undergraduates' predictions of a semester GPA made after an unstructured interview correlated .31 with the actual result against .65 from prior cumulative GPA alone; in Study 3, 169 students ranked the interview types they would want, and random interviews 'were preferred to no interview by 96 participants (57%)'.
5 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.