Do AI Detectors Work on Interview Transcripts or Written Responses?

Do AI detectors work on interview transcripts?

Not well enough to act on. The published error rates come from text long enough to give a classifier something to work with; an interview answer or a short written response usually isn't. OpenAI's own classifier was rated unreliable below 1,000 characters (roughly a 150-word answer), and the company withdrew it in July 2023 for low accuracy ².

The numbers behind that withdrawal are the ones to hold onto. On its own evaluation set, the classifier correctly identified 26% of AI-written English text and labeled human writing as AI 9% of the time, and OpenAI's published guidance was that it should not be used as a primary decision-making tool ². That is the organization with the most data on how these models write, publishing its own ceiling.

Independent testing lands in the same place. A review of fourteen detection tools (twelve free ones plus Turnitin and PlagiarismCheck) concluded that the available tools are neither accurate nor reliable, that they skew toward calling text human-written, and that ordinary obfuscation makes them significantly worse ³.

The 94%-and-up accuracy figures you see quoted come from vendors testing themselves on their own corpora. Ask any of them for three numbers: the false-positive rate on answers under 200 words, on non-native English, and on text a candidate generated and then edited. Those three decide whether a flag means anything, and they are the three nobody publishes. If the live round is what actually worries you, that is a different problem. See what to do about AI help during a live interview.

Why do detector errors cluster by writing register?

Because the thing detectors measure is predictability, not authorship. Text with low perplexity (common words in an expected order) reads as machine-written to a classifier. Plain, formulaic prose is exactly what a competent engineering postmortem, a consulting deck summary or a claims file note is supposed to be, so the error rate lands unevenly across the roles you hire for.

The clearest published evidence of that skew concerns writers rather than genres. Seven detectors were run over 91 human-written TOEFL essays: 61.3% were misclassified as AI-generated on average, all seven agreed on 19.8% of them, and 97.8% were flagged by at least one tool. The same detectors classified US eighth-grade essays by native speakers almost perfectly ¹.

Nothing in that mechanism is specific to language proficiency. It penalizes any writer whose register is constrained by house style, by a template, by a compliance requirement, by training that told them to write short declarative sentences and cut the adjectives. The candidate who writes the way your incident reports are supposed to read is the candidate most likely to be flagged.

The same study showed the failure runs the other way too. Asking a model to enrich the word choices in those essays dropped the average false-positive rate from 61.3% to 11.6% ¹. One prompt moved the result by roughly fifty points, which tells you what the result is measuring. A candidate who wants to clear a detector can; the one who gets caught is the one who wasn't trying to. When a round starts producing suspiciously uniform answers, the polish itself is the thing worth reading, not the provenance.

Does the transcript itself add error?

Yes, before any detector reads it. A transcript is a machine's guess at what was said, and the guess is worse for some speakers than others: across five commercial speech recognition systems, average word error rate was 0.35 for Black speakers against 0.19 for white speakers ⁴. Running a text classifier over that output stacks one uneven error on another.

Consider what a transcription error does to a perplexity score. Every substitution, dropped word and mangled proper noun makes the text less predictable, which pushes a transcript toward "human." Every cleanup the service performs (inserted punctuation, dropped filler words, normalized speaker labels) pushes it back toward "AI." Neither movement has anything to do with the candidate.

Written responses carry a smaller version of the same problem. A candidate typing into your form on a phone, a candidate who drafted in a second language and tidied it up, and a candidate using the grammar checker that ships with their browser all produce text with different statistical fingerprints and identical underlying work. The instrument is reading tooling, not people.

Can you act on a detector flag?

Treat it as a selection procedure, because that is what it is. Any measure used to make an employment decision falls under the EEOC's guidance on tests and selection procedures, and a procedure that screens out a protected group has to be shown job-related and consistent with business necessity ⁵. No detector vendor publishes validity evidence of that kind.

Set the guidance next to the error data and the exposure states itself. The published false positives concentrate on non-native English writers ¹, national origin is a protected class, and a detector score is not job-related to anything: it predicts a text's statistical profile, not job performance. When a candidate asks how the flag was produced, the answer is a proprietary number the vendor will not explain and you cannot reproduce.

What would it take to check a detector for adverse impact? A fixed, inspectable instrument, for a start, and a set of tools an independent review found neither accurate nor reliable is not one ³. The full argument, including which state rules bite first, is in whether it is legal to reject on an AI-detector result.

Replace authentication with evaluation

Stop asking who wrote the answer and start asking what the answer proves. An interview answer is a claim about work, and a follow-up that forces the candidate to operate on their own answer (recompute it, defend the number, name what would make it wrong) cannot be prepared in advance, because it depends on what they just said.

Three changes do most of the work, and none of them need a tool:

• Ask the second question. Take any strong answer and go one level down: which of these numbers would change the recommendation, and by how much? A rehearsed answer has depth of exactly one. Follow-up questions expose understanding faster than any transcript analysis.
• Give them the assistant on purpose. A round that bans AI measures a version of the job nobody has. A round where the model is open, and the candidate has to decide what to hand it and what to keep, produces evidence you can read.
• Score against something written first. Decide what counts as a good answer before the first candidate, apply it to everyone, and record why each answer landed where it did. That record survives a challenge. A detector score does not.

The general version of this trade, dropping the authentication step and moving the burden onto a work sample, is worked through in dropping the resume screen for a work sample.

See a sample report >