AI Detectors Are Reliable Enough for a Question, Not for a Grade

Measured reliability depends on which writing was tested, and the two bodies of evidence disagree for exactly that reason. The 2023 Patterns study ran seven detectors against essays by people still learning English and found error rates too large to rest a decision on ¹. Tests run in 2026, on different text, reported far lower false-positive rates ⁴⁸. So the answer depends on what is being asked of the reading.

The figure that still circulates comes from that Patterns study. Across seven detectors, the average false-positive rate on 91 TOEFL essays written by people was 61.3 percent, and at least one detector flagged 97.8 percent of them ¹. The authors advised against using detectors in evaluative or educational settings, above all when the writer is not a native English speaker ¹. It tested seven tools as they stood in 2023, and a teacher quoting it in 2026 is quoting that year's tools.

The student's version of that finding sits on its own page, written for the writer it lands on.

Detection has improved since then. Nature reported in August 2026 that, according to independent analysts, current detectors call solely human writing human almost all the time, and that Epoch AI found zero false positives for Pangram across 495 human-written texts in July 2026 ⁴⁸. The same report carries the caveat that matters for a purchasing decision: only the companies' own internal figures are current, and those are not checked by outsiders ⁴. A number with no date and no corpus beside it is marketing.

Nothing on this page exempts Human from that test. Its figure arrives dated, and the corpus and the run sit on Human's research page: 0 wrongly flagged by Human's default model of 600 human-written papers it had never been trained on; the ceiling on that is 0.498%. That is Human's own in-house measurement, as of September 2026, on its own held-out sets, not an independent audit, and it describes documents rather than students. This is an estimate from our detector. A flag should be reason to look closer and not verified proof of cheating. A department that quotes the number quotes the corpus and the date beside it, the standard applied above to every other vendor.

A teacher makes four separate decisions after a detector returns a number, and the published evidence supports them unevenly. Read the paper again. Ask the student for process evidence. Change a grade. Refer the matter under the integrity policy. The first two rest on a suspicion, which a reading can honestly supply. The last two rest on a finding, which no detector supplies on its own ⁶.

• Read it again. Supported. Turnitin's 2023 guidance told instructors to use the information to start a conversation rather than to draw a conclusion ³.
• Ask for drafts and notes. Supported, and worth doing whether the number came back high or low.
• Change a grade. Not supported by a reading alone, because published false-positive rates range widely by tool and year, and the 2023 results were unevenly spread across writers ¹⁶.
• Refer for misconduct. Not supported by a reading alone. The University of Pittsburgh's teaching center states that detectors are not reliable enough to act as proof ⁶.

The split isn't about how much to trust the software; it's about what each decision requires. A closer read needs a hunch. A grade change needs a claim about a person, and a measurement of text can't make that claim.

Whether a detector belongs in a course at all is a different question from what its output can carry.

Process evidence answers the question a score cannot. A revision history with timestamps, a folder of notes, an earlier essay by the same student, and five minutes of conversation about the argument all describe how a document came to exist. The detector sees the finished text. And the finished text is the one thing a student can change without changing who wrote it.

The 2026 results make that case stronger rather than weaker. Nature reported that when Pangram tested consumer AI editing of student essays, the model still called the results fully human 41 percent of the time. The company's own technical report gives the figure as 41.37 percent of 14,990 student essays that consumer tools were asked to substantially modify, and on a separate set of lightly polished essays the same model reads 99.45 percent as human ⁴⁹. So a clean reading on an edited draft does not show that no AI tool touched it ⁹. That is worth saying out loud to the teacher who treats a low percentage as clearance and files the paper unread.

Ask early. The same request reads as routine before an accusation and as an accusation after one.

A student who drafted in a tool that keeps history can usually produce it in a minute. A student who drafted elsewhere may have nothing to show, and that absence is not evidence. The request settles less than it seems to, so a teacher who makes it should say in advance what would answer the question.

A Google Docs version history is the cheapest form of this evidence in most classrooms.

Arithmetic is the reason, and it's arithmetic any department can run in a minute. Vanderbilt took Turnitin's claimed 1 percent false-positive rate, applied it to the 75,000 papers it submitted in 2022, and estimated that around 750 papers could have been labeled wrongly ⁷. The estimate assumes the vendor's rate holds on Vanderbilt's own papers, and it is arithmetic on documents rather than students. The university disabled the tool in August 2023 ⁷.

The rate reads as small on one essay. Across a year of submissions it is hundreds of papers.

Vendor rates also carry conditions that fall away in the retelling. Turnitin's June 2023 post restricts its document-level figure of under 1 percent to documents its model reads as 20 percent or more AI writing ³. The same 2023 post gave a sentence-level false-positive rate of around 4 percent, roughly one highlighted sentence in every 25 ³. Pittsburgh records that Turnitin acknowledged a higher false-positive rate in June 2023 and published no replacement number ⁶. A threshold written into a course policy inherits every one of those conditions.

The benchmark literature aims at the claims themselves. The RAID paper notes that many detectors advertise accuracy of 99 percent or better. It then evaluates 12 of the tools available and finds them easily fooled by adversarial attacks, by sampling changes and by generators they had not seen before ².

The false-positive arithmetic is worth running once against a real course roster before any policy names a percentage as a threshold.

A referral needs evidence that exists independently of the detector: a process record the student cannot produce, a conversation that does not match the paper, sources that turn out not to exist, or an admission. The reading can sit in the file as the reason someone looked. None of the results cited here supports a reading as the thing that decides ¹²⁶.

Two universities have written down their doubts about the tool itself. Vanderbilt stated in 2023 that it does not believe AI detection software is an effective tool that should be used ⁷. Pittsburgh's teaching center disabled the detector inside Turnitin and endorses no AI detection tool at all ⁶. It also records that vendor accuracy claims of 98 to 99 percent sit beside high false-positive rates when third parties test the same tools. Many vendors, it adds, decline to publish a false-positive rate for their products ⁶.

Pangram's model card puts the limit in the vendor's own words: the error rate is not zero, and a false accusation carries reputational damage and emotional harm ⁵.

The institutional version of this question lands in the same place.

A score is an input. The process around it is the part that has to hold up.