Multiply the False Positive Rate by a Term's Human-Written Papers, Then Check Its Source

A false positive rate is the share of human-written documents a detector wrongly labels as machine-written. It counts documents, not people, and it's measured on whatever writing the measurer picked. Turnitin stated two such rates in June 2023. At the document level the rate is below 1 percent, and Turnitin states it for documents its model reads as 20 percent or more AI writing ¹. At the sentence level it's around 4 percent ¹.

Those two figures answer different questions.

The sentence rate is the one behind each highlight an instructor reads, because the report highlights sentences. At roughly 4 percent, about one highlighted sentence in 25 was written by a person ¹. On a paper carrying forty highlights, that's one or two passages a student can honestly claim.

The document rate comes with a condition that gets dropped in summary. It is stated for documents the model has already put above the 20 percent mark ¹, so it is not, on its face, the rate for every paper a campus scores. That condition decides whether the multiplication below uses the right base.

Each vendor cited here publishes an overall rate above zero ¹³. That's the practical meaning of the line that no detector is completely accurate.

Multiply the papers an institution scores in a term or a year by the published rate. Vanderbilt did that in August 2023: its students submitted 75,000 papers in 2022, and at a stated 1 percent rate that came to around 750 papers that could have been wrongly labeled, so the university switched Turnitin's AI detector off ². The arithmetic was one of three arguments, beside the bias against non-native English writers and the vendor's silence on how the tool decides ².

The multiplication rests on two assumptions, and a policy file should say both: that a rate measured on the vendor's sample holds on this campus's writing, and that it is applied to the human-written share of submissions, not to every paper scored. The table runs four published rates against three volumes of human-written papers.

Row one is Turnitin's figure at launch, as Vanderbilt recorded it ². Pangram's roughly 1 in 10,000 headline comes next ⁴. Its model card supplies the last two rows: one measured on 62,971 academic documents, one on a million web documents ³. Cells round to the nearest whole paper.

Two cautions travel with every cell. First, a flagged paper isn't an accusation. It becomes one when a person acts on it, and how many do is the policy's choice. Second, watch the denominator. A campus that runs the detector on every submission and a campus that runs it only on suspicion are multiplying very different numbers by the same percentage, and only one of them is entitled to the campus-wide figure.

Vanderbilt's post is three years old, and the detectors it describes have shipped new models since. That's the first reason a rate belongs in a policy with its date. It's still an institutional worked example on the record, alongside the other campuses that turned the detector off.

Three provenances sit behind published rates, and they aren't interchangeable. A vendor claim is measured in-house, on writing the vendor selected, at an operating point the vendor set. An independent test is run by someone with no revenue riding on the answer, usually on a smaller and stranger sample. A ceiling is different again: it states a bound rather than a point estimate.

Vendor claims. Pangram's model card reports 0.0041 percent at its production operating point, measured on a million human-written English web documents, and 0.019 percent on a set of 62,971 academic documents ³. Its own explainer rounds the headline to roughly 1 in 10,000 and says the figure moves with the kind of writing ⁴. That caveat is the useful part. A rate measured on web text is a weak forecast for a first-year composition section; the vendor's academic-writing row is the nearer figure, and it is still measured on the vendor's own sample ³.

Independent tests. Bloomberg Businessweek obtained 500 college application essays submitted to Texas A&M in the summer of 2022, before ChatGPT was released. It ran two detectors over them. Between 1 and 2 percent came back flagged, several with near certainty ⁵. A study of seven detectors, published in Patterns, found an average false positive rate of 61.22 percent on TOEFL essays written by people ⁶. Neither test measured the products whose vendor figures sit in the table, and both were run on samples nobody was selling.

Ceilings. Zero observed errors is a bound, not a rate. A held-out set with no false flags proves only that the true rate sits below some ceiling. How low that ceiling sits depends on how many documents were in the set, so a ceiling quoted without its denominator says nothing.

The one ceiling in this article that comes from Human is a bound of exactly that kind, measured in-house on Human's own held-out set as of September 2026 and not by an independent audit: 0 wrongly flagged by Human's default model of 600 human-written papers it had never been trained on; the ceiling on that is 0.498%. This is an estimate from our detector. A flag should be reason to look closer and not verified proof of cheating. The denominator sits next to the bound on purpose, since a ceiling stripped of it would be one more unlabelled figure for a procurement file to reject.

Nature's 2026 news feature put the blunt version on the record: only the accuracy rates the firms announce from internal testing can be current, and those are not externally verified ⁷. A procurement file should record who measured the rate, on what, and when. Those three questions belong in the detector RFP requirements, not in a hallway after the contract is signed.

Errors don't spread evenly across a cohort. Bloomberg Businessweek reported that the students most likely to be wrongly accused are the ones who write most plainly, including neurodivergent students and students working in a second language ⁵. The seven-detector study found an average false positive rate of 61.22 percent on TOEFL essays and near-perfect accuracy on essays by US eighth graders ⁶.

The same study then ran the experiment that shows what the flag was tracking: it had ChatGPT rewrite those TOEFL essays with richer vocabulary, and the average false positive rate dropped from 61.22 percent to 11.77 percent ⁶. The ideas and the authors were the same and the wording was now machine-edited, yet the detectors flagged the essays less. The detector was reading range of expression rather than provenance, and a campus-wide rate can't see that.

For an administrator the consequence is arithmetic again: a single rate multiplied by a single volume hides the section where the rate is worse. Do the multiplication a second time for the populations that write plainest, such as the ESL composition sections and the students with accommodations. A defensible policy for non-native English writers starts from that second number.

One more asymmetry belongs in the count. A wrongly flagged paper and a missed one don't cost the same. A miss is a grade that was too generous. A false flag can become a conduct file, and it's most likely to be opened on the plainest writers in the cohort ⁵.

Three decisions move with this number. One is the threshold at which a report becomes a case. Another is what evidence has to sit beside a score before anyone calls it a finding. The last is who does the reviewing the flags create, because a rate producing 750 wrongly labeled papers a year is a staffing question before it's an ethics question ².

Write the rate into the policy text with its source and its date beside it. A policy that cites an accuracy headline without a denominator has recorded a slogan. The next cabinet can't check it.

Budget the review capacity the arithmetic predicts. Say the multiplication comes to 200 wrongly flagged papers a year. An appeals process designed for a dozen cases fails in week three of the term, and it fails first on the students least able to push back.

Re-measure on a schedule, since vendors ship new models and the published rates move with them ⁷. An annual review is the floor. A new model version is the trigger. Write the date beside the rate or leave the rate out.

Keep the score subordinate in writing. A percentage is a reason to look, and a detector score alone cannot support a finding, least of all at an institution that has done this arithmetic and knows how many wrongly flagged papers its own volume produces.