Hire for Reading When Nobody Remembers Why the Code Works

What does an unexplained module cost the first time it breaks?

The cost is reading work nobody hired for, and it arrives the first time someone has to change one of those modules. Code nobody can account for runs fine until a dependency moves, a rate limit tightens, or a customer reports a number that is wrong by a little. At that point the team is reading rather than writing.

The next engineer walks into that.

Finding the model-written half is not the task, and the detectors built for it do not do it reliably anyway.

An ICSE 2025 study ran five general AI-content detectors, plus one built specifically for code, against output from four models across three code benchmarks. Accuracy sat mostly below 0.6, which the authors call ineffective ³. The tested code was benchmark functions in Java, C++ and Python rather than repositories, so the study says nothing yet about whole codebases like yours.

What is recoverable is intent. Not which lines a model produced, but which decisions nobody can reconstruct: why the retry loop backs off the way it does, why two services write the same table, why a threshold is 0.87 and not 0.9.

Write that inventory as a list of modules with no owner and no account, ordered by how much money moves through them.

The list is shorter than it feels, and it is the job description you are about to write.

If the person who could have explained it has already left, the requisition is their backfill. Writing the backfill req before the only person who knows leaves is the version of this you get to run early. You did not get that one.

Why does hiring for more AI fluency deepen the problem?

Because generation is what filled the repository, and another fast generator adds to the pile nobody can read. A req that asks for strong AI fluency and fast shipping selects for the half of the work that is already cheap. Nothing in it selects for the person who can open a file with no author, no ticket and no comment, then come back with a defensible account of what it does.

METR randomized 246 real tasks across 16 experienced open-source developers between February and June 2025, on repositories they had maintained for about five years. Allowing AI tools made them 19% slower ².

The same developers had forecast a 24% speedup beforehand, and afterwards still estimated they had gained 20% ².

Sixteen people, one setting, and the finding does not transfer to juniors or to unfamiliar code.

Belief and measurement came apart there, and the gap is the part that travels. So treat a candidate's account of how much AI speeds them up as a belief rather than a measurement, and your team's too.

Ask what they read, not what they generated.

The shortage has a supply side. Reading unfamiliar code beside someone senior used to be the standard first year, and it was assembled out of exactly the small tasks now handed to a model. Where juniors get their reps when AI takes the grunt work is the same shortage seen from the other end.

What is the capability called, and who has measured it?

Program comprehension, and it carries a research literature of its own. In 2018 IEEE Transactions on Software Engineering published a field study by Xin Xia, then at Zhejiang University, and five colleagues, who instrumented 78 professional developers across seven real projects and 3,148 working hours. The developers spent about 58% of that time on comprehension rather than on editing ¹.

The same study found senior developers spending a smaller share of their time on comprehension than junior ones ¹.

O*NET lists the work that depends on it. Among the core tasks for software developers sits "Modify existing software to correct errors, adapt it to new hardware, or upgrade interfaces and improve performance," rated 69 for importance on the page as it stood in August 2026 ⁴.

So the requirement is not invented for the occasion. The occupation's own task list carries the work that needs it, researchers have measured it, and the repository is short of it.

The exercise sits under the same US federal rules as the rest of the loop whether or not it is written down, which is an argument for keeping the record rather than against running the exercise. The Uniform Guidelines, adopted in 1978, define a selection procedure as any measure used as a basis for any employment decision, and say the definition reaches "informal or casual interviews and unscored application forms" ⁵.

A hallway conversation about the codebase that feeds the decision already sits inside that definition. The written exercise at least leaves a record of what was asked, and of who was asked it.

Test comprehension on a module from the repository itself

Use a real module from the unowned list, one whose logic a stranger to the domain can follow. Copy it into a clean repository with its tests and nothing else, then give the candidate ninety minutes and an AI assistant. Ask four questions in writing: what does this do, what does it assume, what breaks when one of those assumptions changes, and what would you delete.

The assumption questions are where to look hardest.

Someone who has done this work names a specific assumption and the specific thing outside the code that would falsify it: a currency, a time zone, an ordering the upstream service never promised. Someone who has not describes the code back to you at a higher level of abstraction.

Before anyone sits down, write the answer key. Two reviewers should be able to fill the same four columns separately and agree. "Understood the code" cannot be scored twice the same way; "named the retry loop's assumption that the operation is idempotent" is a yes or a no.

One study points the same way. In a 2026 paper on open-book exams, engineering students were allowed to use ChatGPT and required to submit their interaction transcripts alongside their solutions. The author reports some of the strongest evidence of reasoning appearing where students evaluated incorrect or incomplete model responses, and concludes that correctness of final answers alone "may no longer provide sufficient evidence of comprehension" ⁶.

The paper covers one course, with no control group and no comparison of grades. Read it as a direction, not as a method anyone has scored for reliability.

A review round is the near neighbour of this exercise and a different test. In a review round that tests what to check, the candidate reviews a diff you wrote with problems planted in it, against a key you already hold. The comprehension exercise hands the candidate a module you did not write and cannot fully explain, which is the condition the hire is walking into.

What has to be explainable before the next merge?

Whatever a person on the team can defend without the model that wrote it. Pair the hire with one standing rule, because a single hire does not stop the repository regenerating the condition: nothing merges unless a named human can say what the change does, what it assumes, and what would make it wrong. Put that in the pull request template rather than in a wiki nobody opens.

The rule needs a name on it or it decays into a checkbox. Naming the human who checks it, or nobody did is the general form, and here it is one line asking who read this and what they checked it against.

Generation genuinely produces more, and that is the argument against everything above. Pooled across three company-run randomized trials at Microsoft, Accenture and an anonymous Fortune 100 firm, covering 4,867 developers, use of an AI coding assistant raised completed tasks by 26.08% ⁷.

The standard error is 10.3%, so the honest range runs from roughly 6% to 46%, and the outcome counted was weekly completed pull requests. Volume, not defect rate and not review burden.

Less experienced developers adopted it more and gained more ⁷.

None of that changes the module. It still runs, it still moves money, and somebody will open it under time pressure inside the year.

What the requisition can change is whether that person can read it. Put the four questions in front of your finalists this round. The next module lands with a name on it.

See the benchmarks >