Interviewing
Decide the Interruption Rule Before Anyone Is Stuck
Decide before the live exercise whether an interviewer may intervene, and hold every candidate to the same rule. The workable version is one scripted line at a stated trigger that returns the clock without naming the error, worded identically for everyone and written into the notes with the minute it was said. Watching a candidate follow a confident wrong answer is the evidence the round exists to collect, so do not rescue on impulse. Break the rule and the note has to say so, and for whom.
The takeThe impulse to speak is almost never about the candidate. It belongs to the interviewer, who can see the wrong turn coming and finds sitting still uncomfortable. That discomfort is the exercise working. A round built to show whether somebody catches a bad answer has to let the bad answer stand long enough to be caught or missed. And nothing stops an unscripted rescue from going to the candidate the interviewer already wants to hire, which is the version that costs a real person a job.
Where Olive fits
Open a role and see what the work shows
Olive puts the overreach inside the work rather than in front of an interviewer deciding whether to speak: an async 40-to-60-minute occupational assignment with an assistant that will complete the whole task if nobody stops it, and a human reviewer who writes or edits six findings with the moment in the session behind each one. The candidate is granted the same report, free, on every tier.
Assess your shortlistWhy does one unscripted nudge change what the score means?
Because it changes the test for one person and leaves it alone for everybody else. The Office of Personnel Management's 2008 guide defines a structured interview by three properties. Every candidate is asked the same questions in the same order, every candidate is rated on a common scale, and the interviewers agree in advance what an acceptable answer looks like 1. All three describe what you ask and how you score it.
None of them describes what the interviewer does while the candidate works.
Material left outside the scored protocol does not stay outside the score. Murray Barrick, Brian Swider and Greg Stewart put 189 accounting students through structured mock interviews and measured the impression one interviewer formed during the small talk, before any structured question.
The impression correlated .42 with the same interviewer's later structured rating, and .25 and .24 when different interviewers supplied the rating 2.
Students in mock interviews are not applicants in a live round, and a correlation is not a decision time. What the study supports is narrower and enough: the rating from the interviewer who sat through the unscored minutes tracked that early impression more closely than anyone else's did.
A nudge carries far more than small talk does. Tell one candidate that the model's third answer looks off and you have handed them the finding the exercise was built to watch for. That candidate's score now measures the exercise plus you.
Comparability is why you vary the data, never the task or the bar. The interviewer's own conduct is part of the task, and a protocol has to write it down like any other part.
An informal label does not change the category. The federal Uniform Guidelines, a 1978 regulation, define a selection procedure as any measure used as a basis for an employment decision, running from paper-and-pencil tests through informal or casual interviews and unscored application forms 3.
Calling the hour a conversation moves nothing outside that definition.
What does a candidate riding a confident wrong answer actually show?
It shows whether the person notices, and when, which a live round sees as it happens and a submission can only suggest afterwards. A confident wrong answer is the expensive failure in this work. The moment you point at it, the observation ends and cannot be restarted, because the candidate now knows something is wrong and roughly where.
A 2023 Harvard Business School working paper reports a field experiment with Boston Consulting Group consultants that included one task chosen to sit outside what the model could do.
Consultants working with GPT-4 were 19 percentage points less likely to reach the correct answer: 84.5% of the control group got it right, against 60% and 70% in the two AI conditions 4.
On this task, training in the tool made it worse. The group given a prompt-engineering overview fell 24 points, and the group given none fell 13 4.
Hold the study to what it was: one task, one sample, and a model nobody runs now. The finding is not that AI makes people worse at their jobs. It is that the people using it could not tell which side of the line the task sat on.
Whether somebody can tell is what the exercise is built to see.
So the wrong turn is the item and the clock is how long the item runs. A candidate who catches it at minute nine and a candidate who never catches it are two different results, and both stay readable only while nobody in the room supplies the catch.
Write the one line you are allowed to say, before the session
Put a trigger and a sentence in the protocol, and cap the intervention at one use per session. The trigger is a stated condition rather than a feeling. The sentence is read the same way to everyone, and it gives back the clock without giving away the finding.
Close to paste-ready:
- Trigger: no visible progress for five minutes. A candidate blocked on the environment gets the environment fixed and the lost minutes logged, and that repair does not use up the line.
- Line: "You have twenty minutes left. Work however you want, and there is nothing further from me until time."
- Log: the sentence used, the minute it was said, and which candidate heard it.
The line must not be a question. "Have you checked that?" is a hint wearing a question mark, and the candidate hears it correctly.
The next candidate, who did not get one, is compared with somebody who did.
Five minutes of stillness is easy to produce once "no visible progress for five minutes" is the published trigger. A candidate who knows the condition can collect the line, and that is the real objection to fixing one in advance.
Even the structured interviews researchers publish use, on average, fewer than half of the available structure components. Julia Levashina, Christopher Hartwell, Frederick Morgeson and Michael Campion analyzed 104 interviews reported across 103 articles published between 1997 and 2010, and found they used an average of 5.74 of the 15 structure components in the framework they coded.
Two evaluation-side pieces were among the least used: detailed notes in 19% of them, and no discussion between interviews in 4% 5.
Those numbers count what researchers built into published studies, so read them as a ceiling rather than a floor. Six of fifteen is not a quality score, and the review treats what further components buy as an open question.
The setup half is a separate decision, made a week earlier: what the candidate may use, what they are told, and what the task plants for them to find. Running a coding or case interview with AI open covers that half.
Which mid-session changes count as accommodation, and which are help?
Draw the line at what the change removes. For a candidate with a disability, an adjustment that clears a barrier the exercise never meant to test can be owed, and comparability survives it. Extended time, a screen reader and a different format can sit on that side of the line, though extended time does not where speed is what the exercise measures. Which side a request falls on is worth settling with counsel.
A hint that supplies part of the answer sits on the other side.
The request needs no particular wording. The technical assistance the EEOC issued in 2022 on assessment software sets the threshold low. An applicant who says a medical condition may make a test difficult to take, or may produce a result the employer finds less acceptable, has already asked for a reasonable accommodation 6.
Naming the ADA is not required, and neither is the phrase itself.
Two limits ride along with that. The low threshold governs what counts as a request rather than what has to be granted. And the document came off the agency's site in January 2025, so it is read here through the Internet Archive.
When a candidate says a tool the round bans is how they manage a disability, route it to whoever handles accommodations and treat it as a question about the round's design.
Everything outside accommodation is help, including the help you would defend as obvious.
Put the sentence and the minute in the notes
Record an intervention the way you record an answer: the sentence used, the minute it landed, and which candidate heard it. A note reading "gave a small hint" is not a record, because a second reader cannot tell whether the hint was the finding. Writing out the actual sentence takes seconds, and it is what keeps the round comparable a month later.
Two people reading that note should reach the same account of what happened in the room.
You will break the rule. Somebody will be three minutes from a result and visibly lost, and the sentence will arrive before the protocol does.
Write the exception down too, in the same detail, and name the candidate who got it.
A recorded exception is a fact the panel can weigh against everything else in the packet. An unrecorded one is a difference between two scores that nobody can see and nobody can argue with.
Trigger rates are worth knowing before a candidate meets them. Running the exercise on the current team first tells you how often five minutes of silence means somebody is stuck, and how often it means somebody is thinking.
The 2008 guide states its definition plainly, in three properties: the same questions in the same order, a common rating scale, and interviewers in agreement on acceptable answers 1. The three say nothing about what the interviewer does while a tool in the room produces a confident wrong answer and the candidate watches it happen. Write the fourth line yourself, in one sentence, before the next session opens.
Then hold to it, starting with the candidate you already want to hire.
Common questions
Should you tell a candidate the model is leading them somewhere wrong?
Not about the wrong turn itself, and anything you do say comes from a rule set before the session that reached every other candidate for the role. A live exercise is watching whether the person notices a confident wrong answer, so naming it removes the observation the round exists to collect. The workable middle is one scripted line at a stated trigger: a fixed condition, identical wording, one use per session, written into the notes with the minute. Deciding in the moment produces help that arrives unevenly, and unevenness is the one thing a comparison cannot survive.
What should the scripted intervention actually say?
Something that returns the clock without returning the answer. A workable line names the time left and closes the door: the candidate hears how long they have and that nothing further is coming. It must not be a question. "Have you checked that?" reads as a hint to the person who receives it, and as an advantage to anyone comparing two sessions later. Write the sentence out in the protocol, read it the same way to everybody, and record which candidates heard it.
Does one small hint really make the round uncomparable?
It makes one candidate's result different in kind, which is enough. The score for a session that included a hint measures the exercise plus the interviewer. The score for the next session measures the exercise. Nothing in the notes usually separates them, so a panel weighs two numbers produced under different conditions without knowing it. The cost is not that the hinted candidate did better. It is that nobody can say afterwards how much of the result belonged to them.
What if the candidate is stuck because the environment broke?
That case belongs in the trigger, written down before the session. A candidate burning twelve minutes on a sandbox that will not start is being measured on your setup rather than their judgment. A protocol that cannot tell the two apart keeps collecting the wrong signal. Name the setup case explicitly as a condition where the interviewer speaks, and treat every occurrence as a repair owed to the exercise. Log it either way, so a panel can see that the session lost twelve minutes.
Is giving a struggling candidate more time an accommodation?
Not by itself. The EEOC's archived 2022 technical assistance names extended time as a possible accommodation for a candidate with a disability. It is not owed where the exercise measures speed, and more time for one struggling candidate is help. The difference is what the change removes. An adjustment that clears a barrier the exercise never meant to test leaves the score meaning what it meant. Help that supplies part of the answer changes it. A candidate does not need to name the ADA or use the phrase reasonable accommodation for a request to count. If a medical condition is raised at any point in the round, route it to whoever handles accommodations rather than deciding it mid-exercise.
References
- 1. Structured Interviews: A Practical Guide ✓ opm.gov Supports the three defining properties this article opens and closes on: the same questions in the same order, a common rating scale, and interviewers agreeing in advance what an acceptable answer looks like.
- 2. Initial Evaluations in the Interview: Relationships with Subsequent Interviewer Evaluations and Employment Offers ✓ homepages.se.edu Supports the claim that an impression formed in unscored minutes tracks the same interviewer's structured rating more closely than other interviewers' ratings: across 189 accounting students in structured mock interviews, the impression formed during rapport building correlated .42 with the same interviewer's structured score, and .25 and .24 when different interviewers supplied it.
- 3. 29 CFR Part 1607 - Uniform Guidelines on Employee Selection Procedures (1978), sections 1607.16(Q) and 1607.3(A) ✓ govinfo.gov Supports the claim that calling a live exercise informal does not move it outside federal selection law: the definition of a selection procedure reaches the full range of assessment techniques through informal or casual interviews and unscored application forms.
- 4. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality (Working Paper 24-013) ✓ mitsloan.mit.edu Supports the 19 percentage point drop on the task chosen to sit outside model capability (84.5% correct in the control group against 60% and 70% in the two AI conditions), and the 24-point and 13-point falls in the coached and uncoached AI groups.
- 5. The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature ✓ doi.org Supports the claim that even published structured interviews use fewer than half of the structure components: 104 interviews from 103 articles published between 1997 and 2010 used a mean of 5.74 of 15 structure components, with detailed notes at 19% and no discussion between interviews at 4%.
- 6. The Americans with Disabilities Act and the Use of Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees ✓ web.archive.org Supports the claim that an accommodation request during an assessment needs no particular wording: saying a medical condition may make a test difficult, or may produce a less acceptable result, is already a request, without mentioning the ADA.
6 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.