Ask a Reference for One Episode, Not an Overall Impression

What turns a reference call into evidence?

An episode. A question naming a bounded piece of work with a rough date on it asks the reference to recount something, and recounting is a different act from evaluating: it does not feel like a judgment the former employer could be held to, and it produces detail a second reader can weigh. Rating questions ask the reference to do the evaluating, which is the one thing a reference does least reliably.

The published question lists are almost entirely rating questions. Strengths and weaknesses, would you rehire, how would you rate them on a scale of one to ten. The competing genre answers a different worry, what an employer may lawfully ask, and stops at dates, title and eligibility for rehire. Neither genre contains a question shaped like an episode, which is why so many calls end with a warm sentence and nothing to write down.

The policy sentence is the predictable result. The researchers who reprinted the old reference-check validity figure in 2016 said in the same paper that it may no longer hold, because in the United States many former employers now release only dates of employment and job titles ¹. That is the environment your script has to work in. A question that asks for a judgment runs straight into that policy. Ask instead what happened on a Tuesday in March, and a former manager will usually answer, because describing a project does not feel like passing judgment on a person.

Structure is the other half, and it is the half that makes two candidates comparable. Federal guidance is plain about the shape: a structured interview format, conducted by phone, with written requests producing low response rates and less useful information ². Same questions, same requisition, same number of calls for each finalist, which is the same argument as running fewer reference calls and structuring them. Without that, you have two conversations that happened to be about two people.

Write three episode questions for the role you have open

Start from the two or three things this role will actually be judged on in its first year, then write one question per thing that names a moment instead of a trait. The test of a good one is whether a reference who has not thought about the candidate in eighteen months could still answer it with a story.

Three that travel well, with the version they replace:

1. "What was the last piece of work of theirs you sent back, and what did you send it back for?" Replaces: what are their weaknesses. It gets a specific answer because it presupposes something ordinary and accuses nobody of anything, and "honestly, I never sent anything back" is itself informative. 2. "Tell me about a call they made without you in the room that you would have made differently." Replaces: how are their judgment and decision-making. It surfaces the boundary the person actually drew around their own authority. 3. "What were they working on in their last two months, and how did that land?" Replaces: why did they leave. It gets you the end of the tenure without asking a question the reference has been told to route to HR.

Then ask one follow-up per answer, and make it the same follow-up: what happened next. Most of the information in a reference call sits one layer under the first answer, which is the same reason a second question exposes what a first answer only claimed in an interview. Stop at three plus follow-ups. A twenty-minute call with three questions produces more than a forty-minute call with twelve.

Ask how the work actually got made

Ask what the person produced themselves and what they assembled, framed as a description of process rather than as a suspicion. "Walk me through how that analysis came together, start to finish" is a question a reference will answer at length. A former colleague who watched the work names the parts that were drafted, the parts that were reused and the parts the candidate built from nothing, because that is simply what happened.

The standard question lists do not carry it, and it is now the question the rest of your process cannot answer. A resume, a cover letter and a take-home can all be produced with an assistant, so the finished artifact does not settle who did the thinking. A person who sat beside the candidate for two years does know, and will say so in ordinary language if the question does not sound like an accusation.

The wording matters more here than anywhere else in the script:

• Ask: "How did that piece of work come together?" and "What part of it did they do that nobody would guess from reading it?"
• Do not ask: "Did they use AI for that?" It is a yes-or-no about a tool, the answer is yes often enough to carry no information, and it teaches the reference that you are looking for something to hold against the candidate.

What comes back is a description of how the work got made, and it carries no more weight than the rest of the evidence you have. When two finalists have produced work of similar quality, the reference call is one of the places to look for who did the thinking behind the output, and it is the weakest of them because it is recollection at second hand.

Keep the words, not the rating

Keep the sentences the reference actually said and drop the five-point scale you were going to convert them into. A rating hides what was said, and it is the step where a call stops being comparable to anything: two people who both wrote 4 out of 5 may have heard opposite stories. A quote survives a debrief with its context attached.

There is direct evidence that the content of an interviewer's notes tracks what happens after the hire. Text-mining post-interview notes on 7,650 hires at one large Chinese technology company, Liu and colleagues found that the number of job-related capabilities an interviewer named in the notes was positively related to later performance and promotions and negatively related to turnover, with a one standard deviation rise in how well the notes matched the job analysis corresponding to roughly a 2 percent rise in performance ³. That is one firm, one country, only people who were hired, and a small effect, and the study never tested the notes against the interviewer's own rating. It still points the same way as everything else here: name the capability, quote the evidence, skip the number.

Free text carries one risk worth stating plainly. Narrative references are not neutral just because they are detailed. Coding 624 authentic letters of recommendation for academic posts, Madera and colleagues found 54% of letters written for women contained at least one doubt raiser against 51% of letters for men, with a wider gap on letters carrying two or more ⁴. Academic letters are not employment references, the gap in those headline percentages is small, and the paper's own test runs on coded doubt-raiser scores rather than on those two percentages. The mechanism is still worth carrying: what a writer chooses to volunteer differs by group before any reader gets to it. Asking every reference the same three questions is a partial answer to that, because it puts the same request in front of every writer.

See what gets scored >