Run a Detector Only Under Five Conditions, Never as Sole Evidence

Because each one closes a hole the others leave open. Take the published error rate: if nobody reads it, it changes no outcome. Human review of a report the student never sees ends in a meeting where only one side has the document in front of them, and an appeal heard by the person who raised the flag is the first opinion read a second time. The five stand or fall together.

Here's the set, with the test that proves each one before anyone signs a contract.

Four of five leaves the gap a hearing will find.

The same list doubles as the skeleton of an RFP. The questions worth asking a detection vendor map onto it one for one. Every answer belongs in the contract, not in a renewal call two years later.

A rate with no sample under it is marketing. Turnitin publishes a document-level false positive rate below 1 percent and says the figure holds only for documents its model already reads as 20 percent or more AI writing ¹. Pangram Labs publishes 0.0041 percent on a million human-written English FineWeb documents, and an English miss rate of 0.3396 percent with a stated interval of 0.3242 to 0.3558 percent ³.

Both numbers are the vendor's own measurement, not an outside audit. Even so, they're more than many vendors offer. According to the University of Pittsburgh's Teaching Center, many detection companies refuse to publish a false positive rate for their products at all, while headline claims of 98 to 99 percent accuracy are common ⁴.

Read the conditions attached to a number, because that's where the number lives. Turnitin's sentence-level rate is around 4 percent, so roughly one highlighted sentence in twenty-five was written by a person ¹. Since 8 July 2024 nothing under 20 percent is scored or highlighted at all, and the indicator shows an asterisk in place of a figure ¹⁴, in a band the vendor had already called less reliable. In 2023 Turnitin also raised its minimum to 300 words, below which a document was not evaluated ²; check the current guide for today's minimum. Every one of those caveats is the vendor's own.

Domain matters too. Pangram's own table puts academic writing at 0.019 percent across 62,971 documents ³. That's a different population from web text, so it's a different number.

Then run the multiplication on the district's own volume. Vanderbilt submitted 75,000 papers to Turnitin in 2022 and worked out that a 1 percent error rate would have mislabelled around 750 of them ⁵. That sum assumes every paper was written by a person and that the vendor's rate carries over to the district's own papers, and it counts documents. A board member will ask for that arithmetic. Which figure matters for a given assignment comes down to the difference between a false positive and a false negative.

A rate describes a population. A hearing has one student in it.

Human publishes the same kind of figure, measured in-house by Human at human.olive.is, as of September 2026, on a held-out set of its own: its detector records 0 wrongly flagged by Human's default model of 600 human-written papers it had never been trained on; the ceiling on that is 0.498%. This is an estimate from our detector. A flag should be reason to look closer and not verified proof of cheating. No outside auditor has checked that count. Like the two vendor numbers above it is self-measured, and because the three were measured on different sets they cannot be compared figure to figure.

A named reviewer who didn't raise the suspicion, with the report open, before any message reaches the student. The US Department of Education's 2023 report on AI in education names human decision-making a top policy priority for educational applications ⁶. Turnitin makes the same point from the vendor side: it does not determine misconduct, and the decision rests with the educator and institutional policy ⁷.

Review, then, isn't a courtesy bolted on top, so put the role's name in the policy. Left unnamed, the reviewer turns out to be the instructor who was already suspicious, and review becomes a formality with a signature on it. Who takes the second read? A department chair or an integrity officer can, and so can a standing committee. The one thing a district can't allow is for the person who noticed the flag to be the person who confirms it.

Turnitin's own instruction is to use a highlighted sentence to start a conversation rather than to draw a conclusion ¹. Pangram Labs states in its model card that false accusations of AI use can lead to reputational damage and emotional trauma, and acknowledges a non-zero error rate in its own model ³. Both vendors are describing a screen. A screen decides where attention goes.

The classroom version of that split is the instructor-level decision about running a detector at all, and the policy version is the sentence a district writes down: a detector score alone cannot support a finding.

The same highlighted document the reviewer read, and a route that can change the outcome. The US Department of Education's 2014 guidance says that whenever a provider maintains a student's education records, the school must be able to give the requesting parent or eligible student access to them ⁹. Whether a detection report counts as an education record is a question for district counsel. A policy can promise access regardless. Without the report, an appeal exists in name only.

FERPA, the US federal student records law (34 CFR 99.20 and 99.21, as the regulation read when checked in September 2026), is narrower than an appeal: it reaches what a record says. Under it, a parent may ask the school to amend a record believed inaccurate or misleading, and may ask for a hearing if the school declines ¹². So may an eligible student, meaning one who is 18 or attending a postsecondary institution ¹³. If the hearing leaves the record unchanged, the parent or eligible student can place a written statement in the record. The school then discloses that statement whenever it discloses the part of the record it concerns ⁸.

None of that reviews the integrity finding or the grade, so the appeal of the finding has to come from the district's own policy. This is a description of the regulation, not legal advice.

Three drafting details decide whether the route works in practice.

• The grounds are written down, so an appeal gets argued instead of pleaded.
• Someone other than the reviewer who confirmed the flag makes the decision.
• The clock starts on the day the student is notified, not the day of submission.

An appeal only the accuser can hear is not an appeal.

One more line belongs in the policy, and it costs nothing: a cleared student's file says cleared. Designing the appeal before the first flag lands costs an afternoon, and settling the FERPA question about sending student papers to a detector belongs in the same contract review.

When the review can't be staffed, when the report can't be shown to the student, or when the intake is a population the tool reads badly. Vanderbilt disabled Turnitin's AI detector in August 2023, citing among its reasons the absence of any published method ⁵. The University of Pittsburgh's Teaching Center went further. It endorses no AI detection tool at all ⁴.

Inside Higher Ed counted at least a dozen universities that have disabled Turnitin's AI detection, Northwestern, Georgetown and New York University among them ¹⁰.

Population changes the rate. In 2023, Stanford researchers tested seven detectors. They were near-perfect on essays by US eighth graders, yet misread more than half of 91 TOEFL essays written by non-native English speakers, an average false positive rate of 61.22 percent ¹¹. A district with a large multilingual intake needs a rate measured on non-native writers and should ask the vendor for one. Pangram Labs reports 0.000 percent on the same 91 essays, its own measurement ³. Ninety-one essays is a small sample and TOEFL essays are one population, so ask for the interval too. A vendor with no such figure leaves the district without a rate for that population, and the 2023 result unanswered.

The published average belongs to the sample, not to the intake.

The failure runs the other way too, and a straight ban skips that part. Two Yale teaching-center directors told Inside Higher Ed that humanizing programs are very successful at reducing or eliminating the AI share a detector reports ¹⁰. A tool whose reading a humanizing program can cut ¹⁰, of a kind the 2023 Stanford study found misreading non-native writers ¹¹, cannot be presented as a fairness instrument on the vendor's word alone.

Switching it off costs something too. Assessment has to change, and that work lands on faculty who didn't ask for it. That's why the universities that turned the detector off are worth reading for what they put in its place.

Neither choice is free. But a detector kept without all five conditions in writing is the hardest of the choices to defend.