Work Sample, Interview, or Assessment: Which Catches AI Dependence?

Which of the three actually catches AI dependence?

None of them, if the only thing you look at is what comes back. Rank the three by how easily a candidate who cannot do the work passes anyway, and the unwatched take-home is easiest, a multiple-choice AI-skills test is next, and a structured interview is hardest without being hard. Dependence is a property of how somebody works, so it shows up only in a record of the work happening.

Treat this as a capability question rather than an integrity one. A candidate who hands the assistant the part they cannot do is not cheating you; they are showing you the shape of what they cannot do yet, and that is the thing worth knowing before an offer. The register matters because it changes the instrument: you are trying to see a working pattern, not to catch someone out.

The fourth row is the honest answer to the headline, and it is a variant of the first rather than a fourth method. A work sample whose acts are visible (the questions asked before generating, the source opened, the direction refused) answers the criterion. The same task with only a file at the end does not. That distinction, not the choice of format, is where the signal lives, and it is the whole argument behind rebuilding what a work sample tests.

Why does high validity not mean resistance to an assistant?

Because validity ranks how well a method predicts performance, not how well it holds up when the candidate has help. In the 2023 re-analysis of the meta-analytic record, structured interviews top the list at a mean validity of .42, with an 80% credibility interval running from .18 to .66 ¹. The spread inside one method is wider than the distance between methods.

That interval is the practical finding, and it is usually read past. "Structured interview" is not one instrument. A shared behavioral bank that every mock-interview tool has trained against sits near the bottom of that range; a bank built from decisions your own team made last quarter sits near the top. If your round is running on the universal questions, the number you are relying on describes a different round than the one you have, which is the whole of what AI coaching does to a question bank.

The second thing the estimates do not carry: none of that research was measuring what happens when a candidate has an assistant open. It was measuring prediction of job performance from a selection procedure. So the ranking is a real answer to a real question, and it is not this question. A method can predict performance well and still be trivially passable by someone with a model running, because those are separate properties.

Use the validity literature the way it was meant: to decide whether a method can support a decision at all, and to justify the round to legal and to the hiring manager. Then run the assistant question separately, method by method, because nothing in the meta-analysis answers it for you.

What does an assistant change about each of the three?

It changes what the output proves, and it raises the candidate's confidence at the same time. In a controlled study, participants with an AI coding assistant wrote significantly less secure code than those without one, and were more likely to believe their code was secure ². That pairing is the thing worth seeing: not the assistant's involvement, but a person who cannot evaluate what it handed them.

• The take-home or unwatched work sample loses the part of the brief that was fully specified. Whatever the instructions pinned down, the assistant can finish, so every submission converges on the same competent artifact and the grading distribution flattens. What remains gradeable is the part the brief deliberately left open: an unstated assumption, a defect in the data, two constraints that cannot both hold.
• The AI-skills test names the wrong construct. A multiple-choice instrument measures declarative knowledge about tools, which the tools answer, and which decays as the tools change. It also tells you nothing about the occupation: knowing what a context window is has no bearing on whether an underwriter opens the survey before binding. Set the bar on the work instead, which is the argument in what an AI assessment should measure.
• The structured interview degrades in a specific direction: toward a better-sounding candidate. In the research an MIT Sloan Management Review piece reports, candidates who prepared with generative AI received higher overall interview ratings than unassisted candidates ³. The answers get more polished and more contextualized, and a rating scale that rewards a well-organized STAR answer rewards exactly that.

The common failure is the same in all three: each one collects a description or a product, and dependence is neither. It is the absence of an act: nothing asked before generating, no source opened, nothing refused, nothing checked against anything outside the conversation. An absence is invisible in a deliverable and visible in a record.

Match the method to what the role's work leaves behind

If the work leaves an inspectable output (code, a model, a query, a denial appeal), run the work sample, because content validity is a short argument when the task is the task ⁴. If the work is judgment under constraint, with a recommendation as the artifact, the interview carries more load, and the Uniform Guidelines say why: a procedure aimed at a mental process cannot be supported primarily on content validity, and judgment is named among the constructs excluded ⁴.

That is the clause most three-way comparisons skip, and it is the reason the right combination changes with the requisition rather than with the company. The Guidelines also ask that the manner, setting, level and complexity of the procedure closely approximate the work situation ⁴. A software task meets that bar easily. A market-entry recommendation is an opinion about the future, and a rubric that scores it as "strong judgment" has named the construct the regulation excludes.

The fix for the second case is not to drop the work sample. It is to move the scoring off the recommendation and onto the acts that produced it: which sources were opened, which assumption was written down before the answer, which figure was recomputed when the first one looked convenient. Those are observable, they are gradeable, and they are what screening for AI judgment in non-technical roles comes down to.

One consequence for the process: a single instrument bought once and applied to every opening will fit some requisitions and misfit others, which is the trade in running one assessment or one per role. Pick per family of work (inspectable output, or judgment under constraint) rather than per department.

Score the criterion, then build the round around it

Write the pass condition down first: a candidate who cannot do this work should not be able to produce a passing submission with an assistant open. Then build the round that makes that true: ask for the record alongside the deliverable, add a short structured follow-up on the candidate's own submission, and score acts a reviewer can point at rather than a verdict on their judgment.

1. Author one real ambiguity into the task. Something the assistant will resolve confidently and wrongly: a defect in the dataset, a claim that only settles when a source is opened, two requirements that cannot both hold. 2. Ask for the working record, not only the output. The intermediate artifact, the exchanges, the thing that was thrown away. State plainly that the assistant is allowed, because a policy nobody believes produces a hidden assistant and no record at all. 3. Follow the submission with 20 structured minutes. Ask what was rejected and why, and what would have changed the answer. Those are answerable only by whoever did the work, and the shape of that round is take-home against live working session. 4. Write the answer key before the first invite. Two reviewers scoring the same submission differently is the failure that stays invisible for months, which is why a rubric two reviewers score the same is the expensive part of building this yourself. 5. Run it beside the current round before it gates anyone. Compare the two on candidates you already decided on, and keep the old round until the new one earns it. The method is piloting an assessment before making it a gate.

None of this delivers certainty, and a round sold internally as certainty will lose its credibility on the first bad hire. What it delivers is evidence about one candidate on one occasion: what they framed, what they checked, what they refused. Olive publishes the six behaviors its reviewers write findings against, if a starting list is useful. See how Olive measures this

See what gets scored >