Teams
Mine Eight Rejections for the Bar the Manager Can't Name
When a hiring manager keeps rejecting screens you passed for thin AI answers, stop asking them to define the bar and pull their last eight rejections instead. Ask them to sort those into close and not close, then to name the sentence in each screen note that would have changed the call. What comes back is usually a different competency rather than a higher one. A common version: the manager reads for whether the person refused something a model produced, and the note records which tools the candidate used.
The takeA manager who turns down eight people and calls all eight thin is not being difficult. They are doing the one thing an intake meeting cannot ask of them, which is applying a standard to real cases instead of describing it in the abstract. The description was never going to arrive. Recruiters keep chasing it because a stated bar feels like agreement and a sorted pile feels like homework, but only the pile answers the question. A manager who cannot write a rubric can usually sort eight files into close and not close in under half an hour.
Where Olive fits
Open a role and see what the work shows
A twenty-minute screen can capture a candidate describing the last thing they threw out; it cannot capture them throwing one out. Olive puts the act itself in front of a candidate as work: a role-grounded assignment, an assistant that will overreach if nobody stops it, and a human reviewer who writes or edits six findings with a timestamped excerpt behind each one, granted to the candidate in the same form the employer reads.
Assess your shortlistWhy won't another intake meeting produce the bar?
Another meeting produces another adjective. A hiring manager who calls an AI answer thin has already given you their honest reading of it, and asking again in a conference room gets a longer version of the same word. What the manager can do is sort.
Judgment about particular people is available to a hiring manager in a way that a written standard is not, and the rejected pile is nothing but particulars.
Disagreement at the line between advancing a candidate and rejecting one is ordinary, and it has been measured.
Ashby examined interviews with more than one interviewer inside its own applicant tracking data and found that around 38 percent of scorecard pairs carry at least one point of difference between the two interviewers 1.
The 2-to-3 line takes nearly half of those one-point gaps, and on Ashby's four-point scale that line is where the yes and the no live 1.
Two scorecards held against each other make a pair, so the figure is a rate of disagreeing pairs rather than a share of interviews. The data carries no outcome measure, so nothing in it says which reader was right.
Ashby also reports that scorecard completion runs higher for candidates who get hired, which makes the written record on rejected candidates thinner than the record on the people who got the job 1.
Preventing the split is intake's job. Settling four things at intake and testing them on three real profiles works before the req opens, and thirty screens later that door has closed. The rejections are what you have instead, and they are better evidence than a second meeting can produce.
Sort the last eight rejections into close and not close
Pull every candidate the manager turned down on AI grounds over the last month, cap the set at eight or ten, and put the screen notes in front of them in one sitting. Ask for two piles, the ones that were close and the ones that were never in it, and do not ask for reasons yet.
The sort is the measurement. Reasons offered before it arrive as the same adjective in longer form.
Then take one file at a time and ask a single question about each: what sentence, had it been in this note, would have changed the call?
The answers come back specific because the files are.
Write each one down in the manager's own words, verbatim, and stop yourself from tidying them. The list you end up with is a specification for what a screen note has to contain, and a specification is worth more than a bar.
Expect the close pile to carry most of the information. Two candidates the manager could have gone either way on produce more usable language in five minutes than an hour spent on the ones nobody would have advanced.
In published structured interviews, the rarest pieces of structure sit on the evaluation side. Levashina, Hartwell, Morgeson and Campion content-analysed 104 interviews reported in 103 articles published between 1997 and 2010, and found the average interview used 5.74 of the 15 structure components in Campion, Palmer and Campion's framework 2.
Detailed notes were present in 19 percent of those interviews, and a rule against interviewers discussing a candidate between interviews in 4 percent 2. The count describes what researchers built into published studies, and partly how fully studies reported it, so read it as a ceiling on practice rather than a floor.
Keep the exercise on the right side of one boundary while you run it.
The manager owns the bar and the recruiter owns the evidence is the division that makes a rejection sort legitimate instead of a renegotiation of the req.
What does a thin AI answer usually turn out to mean?
Thin usually names a missing behaviour rather than a missing amount. The manager wanted to hear a candidate refuse something a model produced, or check a claim against something outside the conversation, and heard instead which tools the candidate has open on a normal Tuesday.
Both answers are about AI. Only one of them is about judgment, and the screen collected the other one.
Naming the behaviour is what ends the argument, and the names are already written down. The AI Fluency Framework, written by Rick Dakan of Ringling College of Art and Design and Joseph Feller of University College Cork, splits fluency into four competencies with three named sub-competencies under each 3.
One of them is Deployment Diligence, defined as "Taking responsibility for verifying and vouching for AI-assisted outputs, including thorough fact-checking, testing for accuracy, and validating claims" 3.
The framework is a taxonomy rather than a rubric: no levels, no scoring anchors, no published proficiency thresholds, and a list its own authors describe as a current version. Borrow the names and never a number.
Ask which one act, from a short list, the close pile was missing. Framing the problem before generating anything. Demanding a source for the claim the decision rests on. Keeping the judgment that should not be handed over at all. Refusing an output on stated grounds. Testing a claim against something the model cannot see.
One of those, named out loud, is a bar. Five adjectives are not.
Once the competency has a name, setting a defensible bar for good enough at AI in a specific role becomes a question with an answer. You can write down what evidence of that one act looks like at the level you are hiring.
Rewrite the screen question, then write down what the screen decides
Change the question before changing anything else, because a bar you cannot collect evidence for is not a bar. If the manager reads for refusal, ask the candidate about the last thing they threw out that a model gave them, what tipped them off, and what they did instead.
Then change the note so it stores the answer rather than a rating: write what you heard, never why you think it happened. The rating is the part the two of you already disagree about.
Notes that name the job's capabilities carry signal about later performance. Shanshi Liu and four co-authors text-mined the post-interview notes written about 7,650 candidates at a large Chinese internet technology company. They found that the number of job-related capabilities named in a note related positively to later performance and promotions, and negatively to turnover 4.
A one standard deviation rise in how closely a note matched the company's own job analysis corresponded to roughly a 2 percent rise in performance, in a correlational study at one firm 4.
Every person in that sample had passed the interview and joined the company, so the study says nothing about what those notes would have predicted for anyone rejected.
A rejection sort runs entirely on the other pile.
Anchors have a modest measured case behind them. Levashina and colleagues report Taylor and Small's meta-analysis of 19 past-behaviour interview studies. Interviews using anchored rating scales showed higher criterion-related validity, .35 against .26, and slightly higher interrater reliability, .77 against .73 2.
Anchored scales travel with whatever else careful teams already do, since that difference was measured between studies rather than inside one, and only on past-behaviour questions.
Then write two lines and send them to the manager. One says what the screen decides by itself. The other says what the screen only records and passes upward for them to weigh.
In 1978 the U.S. federal Uniform Guidelines on Employee Selection Procedures defined a selection procedure as any measure or procedure used as a basis for an employment decision. The same document put "informal or casual interviews and unscored application forms" inside that definition 5. A twenty-minute call that removes people from a pipeline is one of those whether or not anybody scores it.
The Guidelines attach a validation burden only where adverse impact appears, and none of this is legal advice.
The fix decays if the two of you drift apart again. Calibrate on excerpts and a disagreement log rather than on what a 3 means.
Run the sort again after the next ten screens.
If the close pile keeps naming the same missing sentence, the bar is real and the screen now collects it. If it names a different one each time, the specification is what keeps moving. Then the problem is the role, not the screen.
Common questions
How many rejections do you need before a pattern shows?
Eight to ten rejections from the most recent month is enough, and more is usually worse. The set has to be small enough that a hiring manager will sit with it for half an hour and recent enough that they still remember the calls. Pull only the ones turned down on AI grounds, keep the screen notes attached, and include the ones you argued about. If fewer than eight exist, run the exercise on what you have and repeat it after the next batch rather than waiting for a rounder number.
What if the hiring manager refuses to sort the pile?
Do the sort yourself and bring your version to the manager. Predict which of the eight they would call close, write the prediction down before the meeting, then show it. A manager who disagrees with your sort will say why in specifics, because the files are sitting in front of them, and disagreement about a named candidate is the data the exercise was after. Refusal to engage with either version is a different problem, and it belongs with whoever owns the requisition rather than with the screen.
What if the bar really is higher, not different?
A genuinely higher bar shows up as a pass rate, and a pass rate is arguable. Count how many of the last thirty screens cleared the AI question and how many of those the manager advanced. A bar that rejects nearly everyone is a decision about the market the role is hiring in, and it belongs in a conversation about the requisition, the salary and the timeline. A bar nobody can state is a different failure, and mining the rejections is what tells the two apart.
Should the hiring manager just run the screen instead?
Moving the screen does not settle the standard. It relocates the disagreement to whoever reads the manager's notes next. The move also gives up the volume filter that makes the loop affordable, since a hiring manager reading forty screens a week has less attention for each one. Fix the question and the note first. If the manager still wants the call after ten screens on the rewritten question, the disagreement was never about the question.
What belongs in a screen note about AI so it survives the argument?
A note survives the argument when it records what the candidate said they did, in their own words, plus the object they did it to. A note reading "checked the model's citation for the revenue figure against the filing, found the wrong year" can be argued with. A note reading "good AI awareness" cannot, and the second is the shape that produces the rematch. Leave out inferences about why the candidate did something, and leave out anything you did not actually hear. Two people reading the first note reach the same picture, which is what the note is for.
References
- 1. Recruiting Operations Benchmarks | 2026 Talent Trends Report ✓ ashbyhq.com Supports the claim that disagreement between two interviewers on one candidate sits at the decision boundary: around 38 percent of scorecard pairs carry at least one point of difference, nearly half of those one-point differences fall between 2 and 3 on Ashby's 1 to 4 scale, and scorecard completion is higher for candidates who get hired, so the record on rejected candidates is thinner.
- 2. The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature ✓ doi.org Supports two claims: that the missing structure is on the evaluation side (104 interviews from 103 articles published 1997 to 2010 used an average of 5.74 of 15 components, with detailed notes at 19 percent and no discussion between interviews at 4 percent), and the payoff of anchored rating scales reported inside the same review from Taylor and Small's 19-study meta-analysis (.35 against .26 validity, .77 against .73 interrater reliability).
- 3. Framework for AI Fluency ✓ ringling.libguides.com Supports the claim that published vocabulary exists for naming the competency a manager is reading for: four competencies with three named sub-competencies each, including the verbatim definition of Deployment Diligence quoted in the article.
- 4. Predictive Validity of Interviewer Post-interview Notes on Candidates' Job Outcomes: Evidence Using Text Data From a Leading Chinese IT Company ✓ frontiersin.org Supports the claim that a note naming the job's capabilities carries signal about later performance, at 7,650 hired candidates with roughly a 2 percent performance rise per standard deviation of note-to-job-analysis match, and the article's own complication that only candidates who passed could be observed.
- 5. 29 CFR Part 1607 - Uniform Guidelines on Employee Selection Procedures (1978), sections 1607.16(Q) and 1607.3(A) ✓ govinfo.gov Supports the claim that an unscored recruiter screen is a selection procedure, quoting the definition's own inclusion of informal or casual interviews and unscored application forms, with the validation burden attaching only where adverse impact appears.
5 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.