The Humanizer Report

What we found

Rewriting tools that get machine writing past detectors also change what the text says. In this test, the tool whose rewrites lost the most numbers also got past a commercial detector most often. We fact-checked only one of the two tools, so for the other this is a count of numbers kept, not of facts kept. A teacher rarely has the original text, so these changes are hard to see from the page alone.

• Rephrasy, a paid rewriting tool, changed the facts in 29 of the 32 machine-written essays we checked. A reviewing AI model judged each of those 29 distorted: three or more changes that would make a reader believe something different from the original, or a changed central claim. The middle essay had 10 such changes.
• Most of the sources did not survive. The 32 original essays cited 152 references, and 55 were still intact in the rewrites.
• Narratives fared better, not well. Of 16 rewritten narratives, 12 were distorted, with a middle count of 3.5 changes.
• A second tool, Deft, kept far more of the numbers: 88 percent of the numbers in its rewrites of essays, against 33 percent for Rephrasy. We did not fact-check Deft’s rewrites, so this says its numbers survived, not that its facts did.
• Pangram 4.1, a commercial detector, labeled 38 of 80 Rephrasy rewrites of those machine-written essays Human, and 4 of 80 of Deft’s. It labeled none of the 80 originals Human. That is two tools, and it is not a rule about all rewriting tools.
• Our own detector has the same weakness. The standard detector behind the “Verified human” mark read 122 of 136 Rephrasy rewrites of machine-written essays as Human, though it called all 136 originals AI.

How it was done

A humanizer is a service that rewrites text so that detectors are less likely to flag it. We gave two of them, Rephrasy and Deft, machine-written text and kept what came back. The text was essays and narratives written by Claude Sonnet, an AI model, and long papers written by Deft. Deft’s rewrites were finished by 26 October 2026 and Rephrasy’s were made on 26 and 27 October 2026 (UTC).

Four checks followed. An AI model compared 48 Rephrasy rewrites with their originals. A script counted how many numbers and years from each original were still in the rewrite. Pangram 4.1 read the originals and the rewrites. Our own detectors read them too. The fact check and both detectors’ readings are dated 27 October 2026.

What the rewrites did to the facts

On 27 October 2026, six reviewing AI models (Claude Opus) each compared eight or nine rewrites with their originals and listed every change that would make a reader believe something different about the original. A rewrite with no such change is faithful. One or two is drift. Three or more, or a changed central claim, is distorted. The line at three is ours.

The changes in the essays were facts altered (144), dropped (119), garbled (36) and invented (18), and in the narratives altered (44), dropped (33), garbled (12) and invented (21). Of the 152 references in the original essays, 55 were still intact in the rewrites.

How to read this. Each row is a set of Rephrasy rewrites checked against their originals. Read the Distorted column: the higher it is, the more rewrites changed what a reader would believe. The last column is the middle number of changes found in one rewrite.

Limits. It is 48 pairs from one tool, each judged by one AI model, and the line at three changes is ours. The comparison needs the original text, which a teacher does not have, and the 119 dropped facts in the essays cannot be seen without it. Deft’s rewrites were not fact-checked.

We tested the judges. We planted a one-word flip, reversing a direction word, in two rewrites, and both were caught as material. A second AI model from another company re-judged four pairs, agreed with the first on three, and found more changes than the first judge on all four, not fewer.

Which tool kept the numbers

We counted how many of the numbers and four-digit years in each original were still, character for character, in its rewrite.

How to read this. Higher means more of the original’s figures survived. A figure rewritten in words counts as lost, so both columns read low for both tools, and the gap between the tools is the thing to read.

Limits. A number kept is not a fact kept: this counts digits and says nothing about whether the sentence around them still means the same.

What a commercial detector made of them

Pangram 4.1 is a detector sold to schools, publishers and others. We read the first 1,000 words of each text through the web dashboard of our own Pangram plan, on 27 October 2026, one reading per text. We did not use its API. “Labeled Human” is Pangram’s own overall label for the text. “At least half human” counts the texts where Pangram’s breakdown puts at least half the words as human. It is the looser of the two, and we show both. Five human-written control texts all read as fully human.

How to read this. Every text here was written by a machine, so a Human reading is a miss. Read down the Labeled Human column: the originals are almost all caught, and the rewrites are where the detector slips. The Deft rewrites are of the same 80 essays as the first two rows.

Limits. It shows how one detector read these texts on one day, on their first 1,000 words. It does not rate Pangram: the same detector called all 80 original essays machine-written. It says little about how Pangram reads human writing beyond the five controls. A later version may read these texts differently.

We planned 100 essays and 60 long papers, and the rewriting tool’s credits ran out first, so 80 and 46 were read. An independent check found that the texts read match the full set on length, level and Rephrasy’s own score. Within Rephrasy, a more distorted rewrite was not clearly more likely to read as human (a correlation of 0.15 over 22 pairs, too few to call), so the link between changing facts and getting past the detector is seen between the two tools and not inside one.

The readings are ours, taken through Pangram’s dashboard. Pangram has not reviewed them.

Our own detector on the same rewrites

On 27 October 2026 we ran the same texts through our standard detector, the one the “Verified human” mark rests on. No model of ours was trained on these texts. The mark is for words a person wrote, as handed in. Text rewritten by a humanizer does not qualify, but our detector does not catch every rewrite: in this test it missed most of Rephrasy’s rewrites of essays and long papers.

How to read this. Every text was written or rewritten by a machine, so Human is a miss. The Original rows show the detector sees the text before it is rewritten. The Rewritten rows show what rewriting does to its reading. Higher in the last column is worse for the detector.

Limits. These are machine texts only, so the table says nothing about how many human texts our detector flags; the model card has that. It is one run of each tool: Rephrasy at one setting, Deft at its four. Our detector was read at its default setting. The 136 originals are the essays Rephrasy rewrote, and Deft’s 239 rewrites cover a larger set of essays that includes them. The long papers have no Original row, because their originals were read inside a larger set of Deft-written papers. The model card’s figure of 20 of 176 humanized papers called human comes from an older, different set, and the two are not added together.

Limits

It does not show that a given student used a rewriting tool, and it is no basis for an accusation. A flag should be reason to look closer and not verified proof of cheating.

Every text in this report was written by a machine and then rewritten by a tool. We did not test human writing that was rewritten, or a rewrite that a person then edited by hand.

Two tools, one run each, in October 2026. Tools update their models, so these figures may not hold later.

We ran and graded every part ourselves, and AI models did part of the judging. No outside party has checked these results yet.

Where the numbers come from

Every figure on this page comes from one of these files, except the 20 of 176 humanized papers called human, which is on the model card (a different, older set). The Reports page lists the other reports behind this site’s numbers.

• rephrasy_facts.md, 2026-10-27. The fact check of 48 Rephrasy rewrites, the references kept, and the numbers and years kept.
• rephrasy_pangram.md, 2026-10-27. Pangram 4.1 on the originals and on the rewrites by both tools.
• rephrasy_exhaust.md, 2026-10-27. How many texts Rephrasy rewrote, by kind.
• deft_exhaust.md, 2026-10-26. How many texts Deft rewrote and wrote.
• E1_humanizer_scores.md, 2026-10-27. Our standard detector on the same originals and rewrites.
• scripts/rephrasy_pangram.py, 2026-10-26. The rule for the Pangram readings, written before any score: which texts, the first 1,000 words of each, the dashboard and not the API.
• context/sessions/2026-10-26-rephrasy-exhaust-and-pangram.md, 2026-10-27. The run log: the version Pangram reported (4.1), the five controls, the short samples, and when the rewrites were made.

If you make a tool or a detector named here and think we got something wrong, tell us on the contact page. We will correct this page and print the date of the correction.