Pick Which Error the Detector Makes, and Name the Threshold in Policy

A false positive is a flag on writing a person wrote. Machine-written work that comes back clean is a false negative. Detectors produce a continuous score, and a threshold turns that score into a verdict, so the two errors trade against each other at exactly that line. Drop the line and more papers get flagged, the machine-written and the human alike. Raise it and fewer of both get flagged, so more machine text passes.

Vendors that report honestly report the pair. Pangram Labs gives a false positive rate of 0.0041% alongside a false negative rate of 0.3396% in its July 2026 technical report, both from the same evaluation ⁷. In a 2023 post Turnitin put its document-level false positive rate under 1%, with the condition attached in the same post: the figure holds for documents its model reads as 20% or more AI writing ¹.

A number quoted without its partner hides the trade. The same post gave a sentence-level false positive rate of around 4%, so one highlighted sentence in 25 may be the student's own ¹. Twelve highlighted sentences mean something different from what the document percentage on the front of the report suggests.

The false positive, and not because it happens more often. Its cost lands on one named student and travels through a disciplinary process, while a false negative costs a grade nobody contests. Vanderbilt ran that arithmetic in public before switching Turnitin's AI detector off: 75,000 papers went through in 2022, and a 1% error rate against that volume is around 750 papers that could have been incorrectly labeled ⁵.

Running the same sum on a campus's own submission count takes ten minutes. It has to run on the human-written share of the pile and say plainly that the vendor's rate is assumed to transfer, and the step-by-step version is worth having before a committee argues about a percentage.

Two measurements make the cost concrete. Bloomberg Businessweek ran 500 college application essays through two detectors, all of them submitted to Texas A&M University in mid-2022, before ChatGPT was released. The result: 1% to 2% falsely flagged, in some cases with claimed certainty near 100% ⁴. A Stanford study of seven detectors on 91 TOEFL essays written by non-native English writers found an average false positive rate of 61.3%, and at least one detector flagged 97.8% of them ³.

That second finding is the one that decides policy, because the error is not spread evenly across a student body. It concentrates on plain writing, which tracks language background rather than dishonesty. The policy consequences sit in the non-native English writer question.

A missed paper is recoverable at the next assignment. A cleared student still carries the meeting, the delay and the file note.

Name the threshold in the policy before the first flag arrives, and name the office that owns it in the same clause. Every detector turns a score into a verdict somewhere, so a school that chooses no point has accepted the vendor's. Turnitin's own history shows that setting moving: it added an asterisk to AI scores below 20% in May 2023, saying the score is less reliable there ², then withdrew the number there in July 2024 ¹⁰.

The same update raised the minimum document length for a reading from 150 words to 300 ². That is a second operating decision, and one a policy can name.

Pangram Labs publishes the arithmetic behind its document verdict. Human when at least 90% of a document's characters read as human, AI when at least 80% read as machine-generated, Mixed for everything between ⁶. A policy can quote a structure like that directly, because it says what a verdict means instead of asserting one.

A second set of published points gives a procurement file something to set beside that one. At human.olive.is, a detector called Human names three: accusation-safe flags a paper above 15% machine content, standard above 6% (the default), sensitive above 2%. Choosing among them settles less than the names suggest, because one rule sits above all three. No setting turns an AI verdict into a Human one; an AI verdict needs at least 80% of the document to read as machine-written. A threshold moves which papers get flagged, not what the strongest verdict is allowed to mean.

Its error figures are in-house measurements on its own held-out sets rather than an independent audit, as of September 2026: 0 wrongly flagged by Human's default model of 600 human-written papers it had never been trained on; the ceiling on that is 0.498%, and 0 misses in 481 wholly machine-written documents; the ceiling on that is 0.62%. Both ceilings shrink with the size of the held-out set and neither reaches zero. This is an estimate from our detector. A flag should be reason to look closer and not verified proof of cheating.

Three published points and a named default belong in the procurement requirements. A product offering one number nobody can see has written the institution's policy for it.

The policy sentence has to carry three facts: which operating point applies, who may change it, and what a flag is. A workable version reads like this. The institution runs its detector at its highest-threshold setting, the one that flags the fewest papers, on all graded coursework, only the academic integrity office may change that setting, and a flag opens an inquiry that reaches its own conclusion.

Each clause closes a gap that otherwise gets filled in the room. Naming the point stops the threshold drifting per instructor, which is how two students submitting comparable work in the same term end up with different outcomes. An owner on the record stops a mid-case setting change from looking like a procedural fix. The third clause, the status of a flag, is what keeps a score from standing in for evidence.

A policy that sets a percentage without saying which error it bought reads as precision and settles nothing. The case against naming an acceptable AI percentage works that argument through in full.

The sentence also has to survive contact with an appeal. A campus running at a sensitive setting generates more flags, so the appeals route carries more traffic and needs the report, the model version and the date attached to every case. Designing that route is the other half of choosing a point, and a policy that picks the number without staffing the route has done only the visible half.

Pangram Labs writes in its own model card that false accusations can cause reputational damage and emotional trauma, and that its model has a non-zero error rate ⁶. A vendor putting that in public is saying the policy has to do the rest.

Because most of the misses on record are not sitting near the line. Nature reported in August 2026 that when Pangram tested consumer AI substantially modifying human-written student essays, the model still labeled 41% of the results fully human ⁸. In a separate test the research firm Epoch AI found that Pangram (version 3.3.2) failed to flag 30 of 297 author-imitation passages as AI, 10.1% ⁹.

Epoch counts a Mixed verdict as a miss, and GPTZero missed 32 of the same 297 ⁹. Nature's report put the figure at around 8% ⁸.

Text a person reworked reads, to the detector, as a person's writing. Epoch reports that most of its author-imitation misses were clean Human verdicts rather than Mixed ones ⁹. A lower document threshold would not reach those, and it would add flags across the honest submissions in the pile.

The arithmetic runs one way. Taking a campus from a standard setting to a sensitive one raises the flag count against the whole submission volume. On the assumption, which a campus should check against its own archive, that human writing is most of what gets submitted, the added flags fall mostly on human writing, and the machine text recovered is only the band that sat between the two lines.

A second reason to leave the dial where the evidence supports it: Nature's assessment of the field is that only the accuracy rates firms announce from internal testing are current, and that those are not externally verified ⁸. Tuning a threshold against an unaudited figure is a decision with no measurement under it.

The measurement worth having is local. Several hundred known-human papers from the institution's own archive, predating the public release of generative AI wherever the archive allows, as the Texas A&M test used essays written before ChatGPT ⁴, run at each candidate setting. What comes back is the flag rate on the actual student population, broken out by course and language program, and that number is the false-positive half of the trade-off, measured on the population that will pay it.