Do Structured Interviews Still Separate AI-Coached Candidates?

Does AI coaching actually beat a structured interview?

No. It beats the questions, not the structure. In the 2022 re-analysis that pulled most selection-procedure validity estimates down by .10 to .20 points, structured interviews still emerged as the top-ranked procedure ¹. Structure is doing real work, and it keeps doing it. What coaching erodes is the assumption underneath a shared question bank: that the answer was composed by the person saying it.

Two findings sit either side of that. Candidates who used generative AI to prepare received higher overall interview performance ratings than unassisted candidates, and what they delivered was often a polished, contextualized response they were repeating rather than composing ³. And faking is not new: in the study that built the interview faking behavior scale, over 90% of undergraduate job candidates faked something during an employment interview, with the subset closer to outright lying running from 28% to 75% ².

Put those together and the diagnosis is narrower than "the interview is broken." Coaching did not invent embellishment. It removed the effort barrier, and effort was the thing quietly sorting your candidates, separating the ones who had thought hard about the question beforehand from the ones who hadn't. That variance is gone, and it was never the variance you wanted anyway. Every answer arriving polished is a measurement problem, not a character problem.

The structure itself is untouched. Same questions in the same order, a common rating scale, interviewers who agree in advance on what an acceptable answer contains ⁴: none of that depends on the questions being secret.

Why has the universal behavioral bank stopped separating anyone?

Because it is public, finite, and old. Tell me about a time you disagreed with your manager, describe a failure, walk me through a conflict. That list has been printed in every career guide for decades, which makes it ideal training material. A tool that rehearses a candidate against forty of those items is not guessing at your process. It has already seen it.

The failure is quieter than cheating. A coached answer arrives correctly scoped, in the situation-task-action-result shape your rubric was written to reward, with a plausible number in the result. An anchored rating scale rewards exactly those properties, because those properties were once an honest proxy for having done the thing. Now they are a proxy for having prepared with a tool that produces them on request ³.

So the scores compress. Everyone lands at a 3 or a 4 on a five-point anchor, the panel argues about tie-breaks, and the decision quietly relocates to whoever was most likeable in the room, which is the unstructured interview wearing a rubric.

The tell is easy to check against your own data. Pull the last thirty scored interviews for one role and look at the spread per question. A question where nobody scored below the midpoint is not a hard question. It is a question that has stopped asking anything.

Build the question set from last quarter's real decisions

Take six to eight decisions your team actually made in the last quarter (the ones that were argued about) and write one question per decision. The job analysis that OPM's structured-interview guide puts first is exactly this, done at the level of your own field rather than at the level of a competency label ⁴. The bank becomes unguessable because it is local.

The mechanics are in that guide and they are worth following literally. Convene six or seven experienced people who do the job, have them write the questions against the competencies the analysis produced, and keep each question aimed at a specific situation, the actions the person took or did not take, and what those actions changed ⁴. Superlatives do the narrowing: the last one, the worst one, the one you got wrong ⁴. Four to six competencies is the normal load for a single interview ⁴.

Written out by field, the questions look like this:

• Claims: "Tell me about the last file you conceded. What in it made conceding right, and who disagreed?"
• Analytics: "Describe the most recent number you published that turned out to be wrong. How did you find out?"
• Marketing: "Walk me through the last channel you killed. What did you stop believing?"

None of those can be answered from the wording alone, and none of them sit in a mock-interview bank, because they came out of a room. The rubric still transfers (what the rows say for an AI-assisted answer is the same work either way), but the questions do not, and that is the cost you are accepting.

Rewrite roughly once a year, or whenever a question stops producing spread.

Which questions survive coaching, past behavior or situational?

Past behavior, on the only direct evidence available. Levashina and Campion's faking experiment found past-behavior questions more resistant to faking than situational ones. It also found, in the half nobody quotes, that follow-up questioning increased faking rather than reducing it ². Over 90% of the undergraduate candidates in that work faked something during an employment interview ². Coaching did not create this problem. It industrialized it.

That second finding deserves more attention than it gets, because probing is the standard advice. The MIT Sloan piece on AI-prepared candidates recommends five kinds of follow-up: break down the process, ask why the decision led to the outcome, probe the situational limits, ask what alternatives were weighed, ask what the drawbacks were ³. Those are good questions. They are also more narrative surface, and narrative surface is what an embellished story is made of.

The reconciliation is in what a probe asks for. A follow-up that requests more story invites more story. A follow-up that requests something checkable outside the story does not:

• Not "why did that work?" but "what was the number before and after?"
• Not "what would you do differently?" but "who told you it was wrong, and when?"
• Not "how did you approach it?" but "what was the option you rejected, and what did rejecting it cost?"

Names, dates, figures and the losing option are hard to keep consistent across four minutes of an invented account, and trivially easy to produce when the account is real. How far a second question can get has limits, but that is where the remaining separation lives.

Situational questions are not worthless. They are still fine for judgment on a scenario nobody has met. Just do not carry the weight of the decision on them.

What structure still can't reach

A description of work, however well probed, is still a description. A structured interview can capture a candidate saying they would check the model's most quotable claim against the source; it cannot capture them checking one. That gap existed before AI coaching, and coaching widens it, because fluent description of a process is the specific thing a model produces best.

Three limits worth naming to whoever signs off on the round.

The questions are real work and they go stale. Six to eight local questions cost a room of experienced people an afternoon plus a rating scale, and once thirty candidates have been through them, the good ones leak. Budget for a rewrite, not for a build.

Structure is also the thing carrying your legal position. An interview used to decide who gets hired is a selection procedure, and it has to be job-related and consistent with business necessity ⁵. Questions traced to a documented analysis of your own job help you there; a bank someone downloaded does not. The standardization (same questions, same order, agreed answers) is what makes the round defensible, so change the source of the questions and leave the process alone ⁴.

And the format has a ceiling. If what you need to know is whether someone checks a confident claim, a work sample answers that better than a question about it does. If the worry is a candidate reading generated answers during the call rather than before it, that is a different problem with different tells.

None of this means structure is finished. It means the structure was never the part that was secret, and the part that was secret is now public. Move the secret.

See how it works >