Detector, Interview, or Work Sample: Which Defends the Decision?

Which of the three actually defends a hiring decision?

A work sample or a structured interview, depending on the job. Both can be built to produce the kinds of evidence the Uniform Guidelines recognize: criterion-related, content, or construct ¹. A detector produces none of the three. It scores a document for how predictable its wording is, which is not a work behavior, not a job requirement, and not something you can put in front of counsel.

The detector's problem is not that it is inaccurate, though it is. It is that accuracy about authorship is not evidence about the job. What a detector measures is a property of the text, and the measured accuracy of that property is well below what most buyers assume.

The other two are not interchangeable either. A structured interview captures a candidate describing work. A work sample captures work. Both are defensible, and they defend different claims: if the decision rests on "this person can produce X," a description of producing X is a weaker record than X.

What counts as validity evidence for each one?

Three strategies, and only three: criterion-related, content, and construct ¹. A structured interview is usually defended on content (the questions sample the job's work behaviors) or on criterion evidence, if you have enough hires to correlate scores against performance. A work sample is defended on content, and the Guidelines say the closer the procedure sits to the actual work, the stronger that case gets ¹.

That is the whole menu, and a vendor's accuracy figure is not a fourth item on it. One of the most comprehensive public tests of detection tools put 14 of them against human, AI-written, machine-translated, human-edited and paraphrased documents: the best reached 76% overall, and across the field they averaged 30% on human-edited text and 15% on paraphrased text ³. Even at 100% those numbers would say nothing about performance on a job.

Adverse impact is where the detector stops being merely unhelpful. Seven detectors run over 91 TOEFL essays by non-native English speakers returned an average false positive rate of 61.22%, while classifying US eighth-grade essays almost perfectly ⁴. A neutral selection procedure that disproportionately excludes on a protected basis has to be job-related and consistent with business necessity ², and a vendor threshold is not that showing. Run the four-fifths arithmetic on any instrument before it gates anyone.

Interviews and work samples carry adverse-impact risk too. It travels with the content rather than with the writing style: a coding task heavy on unfamiliar tooling, an interview rubric that rewards one idiom of confidence. The difference is that you wrote those, you can inspect them, and you can change them.

Why is a work sample easy to defend in software and hard in consulting?

Because content validity rests on resemblance, and the resemblance is obvious in one case and arguable in the other. The Guidelines ask you to show that the behavior in the selection procedure is a representative sample of the job's behavior, or that it yields a representative sample of the work product ¹. A patch to a repository is a work product. A market-entry recommendation is an opinion about the future.

Judgment is on the regulation's own list of constructs, alongside intelligence, aptitude, personality, commonsense and leadership, and content validity is not an appropriate strategy for procedures that purport to measure traits or constructs ¹. That clause catches most consulting cases by itself. A consulting work sample whose rubric reads "demonstrates strong judgment" has named a construct, and the content argument stops carrying it.

The fix is not to drop the work sample. It is to score observable acts instead of the construct: which sources were opened, which assumption was written down before the recommendation, which number was recomputed when the first answer looked convenient. Those are behaviors a reviewer can point at in the record, and an observable act is the level at which the Guidelines ask a skill to be defined ¹.

Two further clauses are worth designing against. The manner, setting, level and complexity of the procedure should closely approximate the work situation, and content validity is not available for something an employee is expected to learn on the job ¹. So a task sized to a real deliverable defends better than a puzzle, and a task testing tooling your new hire will pick up in week two defends worse than one testing what they must already have. If the format itself is still open, work sample against structured interview is the narrower comparison.

What does each one cost you and the candidate?

The detector costs nothing to run and the most to defend. A structured interview costs interviewer hours, the scarcest resource on most teams, and returns a description of work rather than work. A work sample costs candidate hours, which is the cost that shows up later as attrition, plus reviewer hours to grade. Price the grading, not the task.

Candidate experience separates the two survivors more sharply than cost does. A work sample takes the most candidate time and is the easiest to justify while taking it, because the relevance is visible from the first screen. A structured interview takes the least candidate time and rewards rehearsal: the practiced answer and the earned one look identical in a transcript.

The detector sits in a different category again. A candidate cut by one is never told, cannot appeal, and did nothing wrong. Given where the measured errors concentrate ⁴, the people that happens to are not randomly distributed, which is a fairness problem before it is a legal one.

Two costs get underpriced almost everywhere. Grading is the first: a work sample with no answer key written in advance turns into an opinion poll among reviewers, and the disagreement stays invisible until someone audits it. Accommodation is the second: any timed instrument needs an accommodation path planned before the first invite goes out, not improvised on the third.

Pick the instrument the job's own evidence supports

Write down the job's critical work behaviors first, then pick the instrument that can sample them ¹. If the work leaves a product a reviewer can hold (code, a model, a memo, a denial appeal), run a work sample. If it does not, or if the observable behavior is genuinely conversational, run a structured interview and defend it on the same job analysis.

Then keep four records, because together they are what a defense consists of: the job analysis naming those behaviors, the rubric with its scoring anchors, the log of who scored what, and the reason each behavior is on the list. The Guidelines expect the analysis to focus on work behaviors and the products they result in ¹, and the EEOC expects the procedure to be job-related and consistent with business necessity if it excludes disproportionately ².

Use the detector for nothing. If the worry underneath the question is that take-homes now arrive finished and polished from everyone, that is a real problem with a different answer: change what the task asks for, so that a polished artifact stops being the evidence and the record of producing it becomes the evidence instead. At the level of the role, the same move is hiring for what a person verifies rather than what they can produce.

Olive publishes what it has validated and what it has not. See how Olive measures this

See what gets scored >