Disclosure, Detection, or Observation: Which AI-Use Policy Holds Up?

Which of the three actually holds up?

Observation. Ranked on enforceability, legal exposure and information value, watching a candidate work with AI wins on all three, and the other two only trade places. Detection is unenforceable and carries the worst exposure. Honor-system disclosure is enforceable in the trivial sense that nothing gets checked, carries almost no exposure, and tells you nothing you can act on.

The third column is where most policy drafts go wrong. Disclosure and detection are both attempts to answer a question about a document: was this written by a model. Observation answers a different question entirely, which is the one the hire turns on: can this person work with a model and still be the one making the decisions.

One rule governs all three, and it is worth reading before the policy is drafted. Under the Uniform Guidelines, a selection procedure is "any measure, combination of measures, or procedure used as a basis for any employment decision," and the definition explicitly reaches "informal or casual interviews and unscored application forms" ⁴. A disclosure checkbox that routes someone out of the pile is a selection procedure. So is a detector threshold. So is an observed work sample. Once any of them produces adverse impact, it is discriminatory unless it has been validated ⁴.

So the honest framing is not "which policy is safest." All three sit inside the same rule. The question is which one you can defend once it is in there.

Why detection loses before the policy is written

Because it cannot be made accurate enough to carry a decision, and its errors are not random. Fourteen detection tools tested against human and machine text were found to be neither accurate nor reliable, biased toward classifying output as human-written, and made materially worse by ordinary obfuscation such as paraphrasing ¹. Seven detectors run over 91 human-written TOEFL essays produced an average false-positive rate of 61.3 percent ².

That second number is the one that ends the conversation. In the same study, 97.8 percent of those TOEFL essays were flagged by at least one detector, while essays by US eighth-graders were classified with near-perfect accuracy ². When the TOEFL essays were rewritten with enriched word choices, the false-positive rate fell to 11.6 percent ². The tool is reading register, and register tracks first language, education and profession.

National origin is a protected class, so an error pattern that lands on non-native English writers is the adverse-impact case assembling itself. And a procedure with adverse impact is discriminatory unless validated ⁴. No detector vendor publishes a validation study relating its output to job performance, because there is nothing to relate: the output describes a paragraph, not a person.

The usual rescue is a human in the loop. It does not work, for a mechanical reason. A reviewer who forwards every flag to a rejection has not decided anything, and in New York City the test for an automated employment decision tool is whether it "substantially assists or replaces discretionary decision-making" ⁶. Where that test is met, a bias audit must be done before use and NYC candidates notified 10 business days ahead ⁶. A rubber stamp assists substantially.

More on whether detectors work at all in a hiring context, including the arithmetic of a 1 percent false-positive rate against a real funnel.

When is honor-system disclosure enough?

When AI genuinely isn't part of the job, and the policy exists to set an expectation rather than to gather evidence. A disclosure line costs nothing, reads as respect rather than suspicion, and gives a candidate a clean place to say what they used. It stops being enough the moment the answer starts deciding something.

The failure is not that people lie. It is that the incentive structure guarantees the quiet answer. Microsoft and LinkedIn's 2024 Work Trend Index, fielded to 31,000 knowledge workers across 31 markets, found 75 percent using AI at work, 78 percent of those users bringing their own tools in, and 52 percent reluctant to admit using it for their most important tasks ³. That is people describing their current employer, where the job is already theirs. A candidate answering the same question with an offer on the line has strictly more reason to say less.

So an honor-system question collects the least signal from exactly the people whose AI use you most wanted to understand. It is not a screen. Treat it as a stated expectation and nothing more, and keep the answer out of the advance decision, because a disclosure answer used as a basis for a decision is a selection procedure with no validity evidence behind it, on the same footing as the detector ⁴.

Disclosure does two things well. It tells candidates what the rules are before they spend their evening on your take-home, which is a fairness point rather than a compliance one. And it gives you a written record that the expectation was communicated, which matters later if a dispute arises about what was allowed.

What you can and cannot put in the question is its own problem: see what a disclosure question can legally ask.

What does observation buy that the other two cannot?

The thing the policy is actually about: what the person did. Work samples carry high content validity and high criterion-related validity, applicants perceive them as fair, and subgroup performance differences are generally little or none, though that depends on the competencies assessed ⁵. Watching someone work with an assistant turns an unanswerable authorship question into an observable one.

What becomes visible is behavior, not authorship. Whether the first move went after understanding the problem or straight for the output. Whether a source was demanded for the claim the recommendation rested on, and whether anyone opened it. What was kept by hand and what was handed over. Whether anything the assistant produced was refused, and on what grounds. Whether one claim got tested against something outside the conversation. None of that survives contact with a finished document, which is why detection and disclosure both come up empty.

The honest costs are real and worth stating in the policy. Work samples may be costly to develop in both time and money, may require periodic updating, and can be time-consuming and expensive to administer, since individuals have to observe and sometimes rate performance ⁵. That is the trade: the only option that produces evidence is also the only one you have to build.

Two constraints to write in before anyone builds it. Keep the observation scoped to the assignment itself rather than to the candidate's machine. A policy that reaches further than the work is a monitoring policy wearing an assessment's clothes, and it will be read that way. And if a tool rather than a person produces the judgment, you are back inside the automated-decision rules: New York City's definition turns on machine learning, statistical modeling, data analytics or artificial intelligence generating a prediction or classification that substantially assists the decision ⁶.

If you are deciding what the exercise itself should permit, whether to allow AI on the take-home is the same question one level down.

Write the policy from one question: is AI use part of this job?

Answer that first, per role, before anything is drafted. If the work is done with AI, disclosure produces data you cannot use and detection penalizes the behavior you are hiring for, so observation is the only coherent policy. If the work genuinely isn't done with AI, a one-line disclosure expectation is enough and nothing more is worth its cost. The answer is per requisition, not per company.

• Write the answer down for each role, with a reason. "Analysts here draft with an assistant and check the numbers by hand" is an answer. "We're an AI-forward company" is not, and it will not survive a question from counsel. Start from what AI actually does in the role.
• Name the decision each policy element feeds. A disclosure question that feeds nothing is a courtesy, and should be written as one. A disclosure question that feeds an advance decision is a selection procedure and inherits every obligation that comes with it ⁴.
• If a detector stays in the process, demote it to routing. A flag sends a file to a human read and never rejects on its own, and the reviewer records the job-related reason they gave. If nobody can name a case where the reviewer overruled the flag, the tool is deciding ⁶.
• Scope observation to the work. Say what is captured, for how long, and what is never looked at. Declining should be a supported outcome with a stated consequence, not a silent disqualification.
• Write the rejection reason before you write the question. A reason that existed before you met the candidate is the one that holds up when they ask for it, and they will. See how to explain an AI-related rejection.
• Keep impact records per role. Adverse impact attaches to a procedure as used, and a single policy applied across ten requisitions is ten exposures, each with its own applicant pool ⁴.

The rest of the document (scope, retention, who reviews, what happens on a dispute) is ordinary policy drafting. What belongs in an AI hiring policy covers the sections most first drafts leave out.

Read the evidence >