Should You Disqualify a Candidate for Using AI on a Take-Home?

When does disqualifying actually make sense?

Three cases: the brief said not to before they started, the role genuinely forbids the tool, or they claimed work as their own when asked about it directly. Everything else is a standard you invented after reading the submission and applied to one person, and it will not survive the first candidate who asks why. Write the rule for the next round instead.

The second case is real but narrow. Classified or export-controlled work, a client contract that bars third-party processing, a data policy no approved model satisfies yet. Those are conditions of the job, and they belong in the brief in one line. Outside them, banning the tool at the assessment while the team uses it daily tests something the job never asks for.

Your own leadership is probably on the other side of this. In Microsoft and LinkedIn's 2024 Work Trend Index, a survey of 31,000 knowledge workers across 31 markets, 66% of leaders said they would not hire someone without AI skills and 71% said they would rather hire a less experienced candidate who has them than a more experienced one who does not ¹. A disqualification for tool use, in that room, is a decision the hiring manager reverses on second reading.

Whatever you decide, the rule has to exist before the submission does and reach everyone who got the same brief. A standard enforced on the candidate whose writing struck you as suspicious, and not on the one whose style reads as normal to you, is a different test given to different people. It is also the version that becomes expensive when they ask for the reason behind the rejection and the honest answer is that you guessed.

How sure are you that they used AI?

Less sure than the submission feels. Unless they told you, or the file carries a receipt (a pasted chat, a stray placeholder, a leftover instruction), you are reading style and calling it evidence. Clean structure, even paragraph lengths and confident hedging are also what a careful writer produces on a third draft, and what an entire professional register produces every day.

Detectors do not close that gap. Seven of them, run over 91 TOEFL essays written by non-native English speakers, produced a 61.3% average false-positive rate: 97.8% of those human-written essays were flagged as AI-generated by at least one detector and 19.8% by all seven, while the same tools were near-perfect on essays written by US eighth-graders ². The error is not evenly distributed. It falls on people writing in a second language, and national origin sits directly behind that, which is not a distinction you want a rejection resting on. Whether AI detectors work at all in hiring is its own question, and the answer has not improved.

So treat "clearly used AI" as a hypothesis, and test it the cheap way. The artifact will not settle it. Ask about the work: which figure the recommendation rests on, where it came from, what the second-best option was and why it lost. Someone who did the thinking answers in specifics inside a minute, and someone who did not moves to generalities and stays there.

What does unchecked AI actually cost in your field?

That is the question worth answering, and the answer changes by role. The failure mode is not an assistant writing something. It is a confident wrong thing surviving to a place where nobody catches it. In marketing copy the cost is an edit. In a valuation, a denial appeal or a merged pull request, the cost is a decision made on a number nobody opened.

The errors that get through are the near misses. In Stack Overflow's 2025 developer survey the most-cited frustration with AI tools was "AI solutions that are almost right, but not quite," named by 66% of respondents; roughly 46% distrust the accuracy of AI output against 3.1% who highly trust it ³. Almost-right is the expensive class. It survives a read-through and fails downstream, which is exactly the material a polished take-home is made of.

The person holding the output is also the worst judge of it. In a Stanford study of developers writing security-relevant functions, participants with access to an AI assistant wrote significantly less secure code than those without, and were more likely to believe their code was secure ⁴. Confidence moved the wrong way. "The submission looks fine" is not a check, and neither is a candidate's own account of how carefully they read it.

What unchecked looks like, by the field you are actually hiring for:

• Marketing. The most on-message statistic in the brief comes from a vendor's own 40-person customer survey, written up as market-wide. Cheap to catch, cheap to fix, embarrassing in a deck to a client.
• Financial analysis. A growth figure the filing behind it does not support, or a multiple applied to the wrong base. It moves a recommendation, and the reader has no way to see it.
• Software engineering. Code that passes the tests it was handed. The fixture agrees with the bug.
• Legal operations. A citation that resolves to a real case and a paragraph saying something else. Retrieval is not verification.
• Data and analytics. A fluent explanation of a result the dataset does not support, restated more carefully each time you push on it.

Grade against that column, not against tool use. A candidate who generated a first draft and then re-added the segment against the filing did the job. One who wrote every word by hand and shipped an unchecked number did not.

Grade the submission again, then ask about it

Read it a second time for the checks rather than the tool. Four questions answer off the artifact: which claim is the deliverable resting on, what it was checked against, what is cited but plainly never opened, and where a range has been reported as a point estimate. Those separate a candidate who ran the check from one who typed a confident sentence about it.

Polish on its own tells you nothing, and grading a take-home when every submission comes back polished is the general version of this problem. Two graders should be able to fill those four columns separately and agree. "Showed good judgment" cannot be scored twice the same way; "reconciled the segment figure to the 10-K" is a yes or a no.

Then book twenty minutes and ask about their own submission. This is where the disqualification question dissolves, because the conversation gets you evidence and the accusation gets you an argument. The Uniform Guidelines hold that content validity runs to the extent the procedure is a representative sample of the content of the job, and that a procedure resting on inferences about mental processes cannot be supported by content validity alone ⁵. "Did they cheat" is an inference about a mental process. "Walk me through how you settled the discount rate" is the work.

The same three rungs that make follow-up questions expose real understanding apply here: specify, invert, falsify. Ask what the number would have to be for the recommendation to flip. Ask what they could not settle in the time. Ask the same set of every candidate who got that brief, so the round stays comparable and the record shows it. See how Olive measures this.

Write the rule before the next candidate

One line in the brief settles most of this: whether AI is allowed, what has to be disclosed, and what gets graded. Say the tool is permitted, ask for a short note naming what was generated and what was verified, and tell them the note is part of the deliverable. A rule nobody stated is not a rule anyone broke, and a candidate reading your brief has no way to guess at it.

The three lines, verbatim enough to paste:

• AI use is allowed on this task.
• Include a short note: what you generated, what you verified, and how you verified it.
• The note is graded. The polish is not.

Then change what the task asks for, because a brief that wants a finished artifact now measures whose assistant was better. Hand over source material with one thing wrong in it that only checking catches, ask which claim was checked and what changed as a result, and cap the whole thing at an hour. Whether to allow AI on the take-home at all should follow what the job permits: an agency where every draft starts in a model and a bank with a locked-down policy are answering different questions.

The hour between sending the brief and reading the deliverable is the part you still never see. A verification note is written afterwards by the person it describes. It leaks too: a dozen candidates in, the packet is posted somewhere and the task measures recall. Budget the second case at the start, keep one rubric across both, and treat the take-home as one observation from one afternoon, which is what it has always been, with or without an assistant in the room.

See what gets scored >