Research
Model card
The models that are live, what they were tested on, what they measured, and their limits in full.
The models that are live
The detectors’ own health checks are the authority on which models are live, and the status page reads them live, so the two can be checked against each other.
The standard detector decides "Verified human". The students detector reads every text the standard detector calls human; under today's rule its reading is a note and does not decide the mark. The humanizer model reads a text only when you choose it for a single-model reading in the app.
Every number below names its model. The standard detector's default line moved from 10% to 6% on 2026-09-15.
Intended use
Checking English academic and student writing, giving "Verified human" when the standard detector reads it as human, and marking the passages that read as AI-written so a person can look closer.
A flag should be reason to look closer and not verified proof of cheating. It is not intended as the only basis for an academic, disciplinary, employment or legal decision about a person, or for automated penalties of any kind.
Out of scope
- Text under 50 words, which it does not score.
- Languages other than English.
- Kinds of writing it was not trained or tested on, such as fiction, email, social posts, code and business documents.
- Who wrote a text, or how. The mark covers a text on a date.
- Plagiarism, images and web addresses. It checks none of them.
What it was tested on
Every human-written document in testing was written before 2022, so it is known to be human, with one exception. The ELLIPSE learner essays are dated only by their public release in August 2022, three months before ChatGPT’s release. They were never used in training.
The 481 wholly machine-written held-out documents were written by AI models used in training, or rewritten by our own tools: 236 came from the same eight AI models used in training, and 245 were AI text rewritten by our own rewriting tools, not by paid humanizer services. They are long documents.
The release test read 600 human-written papers the standard detector was never trained on.
Which AI models it has been tested against
Fourteen current AI models from three vendors have a measured row on the standard detector. They wrote 4,084 fresh documents, drawn from 310 distinct source papers, across 24 federal fields of study. None was called human-written and two were called partly machine. The 95% ceiling on that miss rate is 0.073% counting documents and 0.96% counting source papers.
Evaluated is not the same as trained on. Three of the fourteen were fixed, before any result existed, as models with no writing in any training or development window. Their 900 documents, from 242 source papers, two vendors and 24 fields, are the part that speaks to a model the detector has never met. None was missed.
The loss that travels with that coverage: on 176 sealed documents from two paid humanizer services, 19 still read as human-written under one reading mode and 20 under the other. A newer test is harder on the standard detector. After one humanizer tool rewrote machine-written texts, it read 231 of 255 long papers and 122 of 136 essays as human-written, but only 98 of 312 narratives. The two tests use different texts and are not added together.
The standard detector: measured results
| Measurement | Result | 95% ceiling | Date | Source |
|---|---|---|---|---|
| Human-written papers it was never trained on, flagged, release test | 0 of 600 | 0.498% | 2026-09-21 | docs/reports/ship_decision_2026-09-21.md §1 |
| Held-out human papers, flagged | 1 of 549, as Mixed | 0.861% | 2026-09-21 | docs/reports/sensitivity_points_2026-09-21.md §2 |
| Real student essays, flagged | 5 of 9,305, all Mixed, none AI | not applicable | 2026-10-27 | docs/reports/t7_human_controls.md; docs/pivot/evals/E2_cascade_cut.md |
| Wholly machine-written release papers, missed | 0 of 220 | 1.35% | 2026-09-21 | docs/reports/ship_decision_2026-09-21.md |
| Wholly machine-written held-out documents, missed, at every setting | 0 of 481 | 0.621% | 2026-09-21 | docs/reports/sensitivity_points_2026-09-21.md §2 |
| Papers from three AI models it never trained on, called AI (edit reading) | 183 of 183 | not applicable | 2026-10-23 | docs/reports/students_protected_2026-10-23.md |
| Documents from 14 AI models, called human-written | 0 of 4,084, and 2 called partly machine | 0.073% per document, 0.96% per source paper | 2026-09-21 | docs/reports/v19b_permodel_2026-09-21.md |
| Of those, by models with no text in training, missed | 0 of 900 | 0.332% per document, 1.23% per source paper | 2026-09-21 | docs/reports/v19b_permodel_2026-09-21.md |
| Reserve documents, scored once after the model was frozen, missed | 0 of 419, and 0 called partly machine | 0.712% per document, 1.98% per source paper | 2026-09-21 | docs/reports/v19b_permodel_2026-09-21.md |
| PELIC learner essays, flagged (edit reading) | 0 of 1,195 | not applicable | 2026-10-23 | docs/reports/students_protected_2026-10-23.md |
| ELLIPSE learner essays, never used in training, flagged, both reading modes | 0 of 3,631 | 0.0825% | 2026-09-23 | docs/reports/parity_2026-09-23/scoreboard.md, ELLIPSE row |
| Partly AI-edited papers called clean, standard setting | 38 of 140 | not applicable | 2026-09-21 | docs/reports/sensitivity_points_2026-09-21.md §2 |
| Sealed paid-humanizer papers called human-written | 20 of 176 (19 under the edit reading) | not applicable | 2026-09-21 | docs/reports/sensitivity_points_2026-09-21.md §2; docs/reports/parity_2026-09-23/stage3_result.md |
| Long machine-written papers rewritten by one paid humanizer tool, called human-written | 231 of 255 (233 under the edit reading) | not applicable | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written essays rewritten by the same tool, called human-written | 122 of 136 (122 under the edit reading) | not applicable | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written narratives rewritten by the same tool, called human-written | 98 of 312 (99 under the edit reading) | not applicable | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
Our standard detector at the standard setting (6%), unless a row says otherwise. The three rows on one tool’s rewrites are machine texts only, so they say nothing about human writing; the sealed 176 are a different, older set. The 95% ceiling is the highest error rate still believable at 95% confidence given what we saw. It is the number to plan against, not the count.
The students detector
Our students model reads every text the standard detector calls human. Under today's rule its reading is shown as a note and does not decide the mark.
| Measurement | Result | Date | Source |
|---|---|---|---|
| Real student essays, flagged | 4 of 4,816, all Mixed | 2026-10-23 | docs/reports/v112_t7_readout.md |
| PELIC learner essays, flagged | 4 of 1,195: 2 AI, 2 Mixed | 2026-10-23 | docs/reports/students_protected_2026-10-23.md |
| Human-written release papers, flagged | 1 of 600, as Mixed | 2026-10-23 | docs/reports/students_protected_2026-10-23.md |
| Held-out human papers, flagged | 2 of 549, as Mixed | 2026-10-23 | docs/reports/students_protected_2026-10-23.md |
| Held-out rewrites by one paid humanizer tool, called AI | 39 of 41 | 2026-10-23 | docs/reports/v112_t7_readout.md |
| Rewrites by a second paid humanizer tool on prompts it never trained on, called AI | 42 of 44 | 2026-10-23 | docs/reports/v112_t7_readout.md |
| Longer machine-written essays rewritten by that second tool in the newer test, called human-written | 82 of 239 (82 under the edit reading) | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Long machine-written papers rewritten by one paid humanizer tool, called human-written | 218 of 255 (218 under the edit reading) | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written essays rewritten by the same tool, called human-written | 103 of 136 (103 under the edit reading) | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written narratives rewritten by the same tool, called human-written | 52 of 312 (52 under the edit reading) | 2026-10-27 | docs/pivot/evals/E1_humanizer_scores.md §1 |
It trained on other rewrites by both humanizer tools, so a tool update can undo the two rows on those rewrites. Both are small sets. The four rows from the newer test (three on one tool’s rewrites, one on the second tool’s rewrites of longer essays) are machine texts only, so they say nothing about human writing; it was not trained on those texts, and they are a different set from the two rewrite rows above, so the two are not added together. Read the second tool’s 42 of 44 and its 82 of 239 side by side: the first is student-register essays on prompts it never trained on, the second is longer essays written by an AI model and then rewritten.
The humanizer model
Our humanizer model is offered in the app beside the standard detector for a single-model reading, never in its place and never for the mark. It runs at the standard setting only. It failed two of the rules it was tested against before release, the two that count wrongly flagged human writing.
Every population this table does not list has no reading on the humanizer model. Not measured is not the same as zero.
| Measurement | Humanizer model | Standard detector | Source |
|---|---|---|---|
| Human-written papers of the release test, flagged | 2 of 600 (ceiling 1.05%) | 0 of 600 (ceiling 0.498%) | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/ship_decision_2026-09-21.md §1 |
| Held-out human papers, flagged | 2 of 549 (ceiling 1.14%) | 1 of 549 (ceiling 0.861%) | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/sensitivity_points_2026-09-21.md §2 |
| A further 1,200 human papers never read before, flagged | 2 of 1,200 (ceiling 0.52%) | not on this card | runs/parity_loop/stage3_readout/readout/readout.json |
| ELLIPSE learner essays, flagged | 0 of 3,631 (ceiling 0.0825%) | 0 of 3,631 (ceiling 0.0825%) | runs/parity_loop/stage3_readout/readout/readout.json; docs/reports/parity_2026-09-23/scoreboard.md |
| Paid-humanizer papers called human-written, of 176 | 7 (8 under the verdict reading) | 19 (20 under the verdict reading) | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/sensitivity_points_2026-09-21.md §2 |
| Long machine-written papers rewritten by one paid humanizer tool, called human-written, of 255 | 212 (219 under the verdict reading) | 233 (231 under the verdict reading) | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written essays rewritten by the same tool, called human-written, of 136 | 88 (91 under the verdict reading) | 122 under both readings | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Machine-written narratives rewritten by the same tool, called human-written, of 312 | 97 (86 under the verdict reading) | 99 (98 under the verdict reading) | docs/pivot/evals/E1_humanizer_scores.md §1 |
| Rewriting-tool outputs called human-written, of 70 | 0 | 5 | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/sensitivity_points_2026-09-21.md §2 |
| Wholly machine-written documents of the release test, missed | 0 of 220 (ceiling 1.35%) | 0 of 220 (ceiling 1.35%) | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/ship_decision_2026-09-21.md |
| Documents by models with no text in training, called human-written | 0 of 900 | 0 of 900 | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/v19b_permodel_2026-09-21.md |
| Partly AI-edited papers given the verdict the record expects, of 140 | 125 (124 under the verdict reading) | 123 under the verdict reading | docs/reports/parity_2026-09-23/stage3_result.md; docs/reports/sensitivity_points_2026-09-21.md §2 |
Humanizer model, measured 2026-09-30 under both reading modes, standard setting. Where the two modes agree one figure is given. The decision to offer it: docs/reports/parity_2026-09-30_two_models_decision.md.
The deep check: measured results
The deep check runs beside the verdict and changes nothing in it. These are its notes on honest writing, so a reader knows how much weight a note carries. A reviewer here means an AI model with web search, reading what the check marked.
| Measurement | Result | Date | Source |
|---|---|---|---|
| Published papers where the reference check marked at least one reference as not found | 7 of 728 | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Of the 31 references it marked, found by a reviewer to exist or not to be references | 27; 3 not found, all in one paper; 1 unclear | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Published papers where the pairs check marked at least one pair of numbers, names or dates | 49 of 728 | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Pairs from human writing that a reviewer found to be a real contradiction | 0 of 40 | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Real and humanized texts where the stock-phrase check found a phrase from its list | 0 of 29,266 (it fired on all 20 test texts planted with listed phrases) | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Humanized texts the standard detector read as human that the deep check would have held back | 0 of 130 | 2026-10-28 | docs/pivot/evals/E3_evidence_checks.md |
| Wikipedia check of dates and numbers: planted errors found, and true sentences marked as differing | 4 of 5; 0 of 15 | 2026-10-28 | context/sessions/2026-10-27-pivot-l2.md |
The reference and pairs checks mark honest papers too, so each note is a prompt to look, not a finding that a text is wrong. Only 107 of 26,220 human texts kept a reference list, so the reference rows say little about a full student reference list. The Wikipedia row is 5 planted errors and 15 true sentences, too few for a rate. All the human writing tested is dated before 2022.
The three settings
| Accusation-safe | Standard (default) | Sensitive | |
|---|---|---|---|
| Flags a text when more than this share reads as machine-written | 15% | 6% | 2% |
| Human-written release papers flagged, of 600 | 0 (ceiling 0.498%) | 0 (ceiling 0.498%) | 1 (ceiling 0.788%) |
| Held-out human papers flagged, of 549 | 0 (ceiling 0.544%) | 1 (ceiling 0.861%) | 2 (ceiling 1.14%) |
| Wholly machine-written documents missed, of 481 | 0 | 0 | 0 |
| Partly AI-edited papers called clean, of 140 | 48 | 38 | 38 |
| Rewriting-tool outputs called human-written, of 70 | 6 | 5 | 4 |
| Paid-humanizer papers called human-written, of 176 | 25 | 20 | 14 |
| Long machine-written papers rewritten by one paid humanizer tool, called human-written, of 255 | not measured | 231 | not measured |
| Machine-written essays rewritten by the same tool, called human-written, of 136 | not measured | 122 | not measured |
| Machine-written narratives rewritten by the same tool, called human-written, of 312 | not measured | 98 | not measured |
Our standard detector, re-fitted 2026-09-21, source docs/reports/sensitivity_points_2026-09-21.md §2. The last three rows were read at standard only (docs/pivot/evals/E1_humanizer_scores.md §1). A verification always reads at standard. The humanizer and students models run at standard only and have no reading at the other two settings.
Limits
- Zero observed is not zero forever. The ceiling, not the count, is the number to write into a policy.
- The false-flag figures hold only for English academic and student writing like the test sets.
- Every human-written document we tested was written before 2022, except the ELLIPSE learner essays, dated only by their public release in August 2022. Writing from after AI tools became common has not been tested.
- ELLIPSE carries no writer identifiers, so its count is per essay, not per writer.
- No grammar-checking product was tested, so we cannot say whether a paper written with one will be flagged or not.
- Paid humanizer tools can get text past the standard detector: it called 20 of 176 humanized AI papers human and, in a newer test of one humanizer tool, 231 of 255 rewritten long papers and 122 of 136 rewritten essays human, so a humanizer tool can get text past it.
- It does not answer under 50 words, and readings under 150 words are weak. Machine-written text under 300 words has not been measured.
- A count of zero false flags on learner essays is not proof of fairness.
- The marked passages are a reading. Their placement accuracy has not been measured on this run, and no number on a result is the probability that a person used AI.
- We run and grade our own tests. No outside party has checked these results yet.
- The first check after a quiet spell can take longer. There is no load test.
Ethical considerations
A published error rate helps a school write a policy and answer an appeal. It also says plainly that an error will eventually happen to someone.
A flag should be reason to look closer and not verified proof of cheating. It does not show intent.
Ask us how pasted text is handled: hello@olive.is.
Check your writing yourself.
Free with a school email, up to 50 papers a month.