How AI Use Belongs in a Performance Review, and How It Doesn't

Why an AI-usage rating goes wrong

Because the only thing an AI-usage rating can measure cheaply is self-report, and self-report about AI is unreliable in a way that has been measured. Sixteen experienced developers in a randomized trial forecast a 24% speedup from AI tools, believed afterwards they had gained 20%, and were in fact 19% slower across 246 real tasks ¹.

Two things follow. The first is that a rating built on what people say about their own AI use is measuring something other than what the person can do. In a study of 288 teachers in Taiwan who sat both a self-report and a knowledge-based test of AI literacy built on the same framework, correlations between the self-reported and objective factors ran from r = 0.07 to r = 0.24, and the profiles found included people who overestimated themselves and people who underestimated themselves ². Teachers are not employees, and a weak correlation is not proof that anyone is bluffing. It is still a poor foundation for a number that follows someone into a compensation conversation.

The second is that the prize is smaller than the line item implies. A nationally representative US survey asked people who use generative AI at work how much longer the previous week would have taken without it: mean self-reported savings came to 5.4% of work hours among users, which the authors put at 1.4% of hours across all workers ³. Those are self-reports carrying the same caveat. An average that small, measured that softly, should not be carrying a rating.

A separate cost lands in the work itself. Price a method and people optimize the method. The person who tried a model on a task it handles badly, noticed, and went back to doing it by hand has done the job right, and an AI-usage scale marks them below the colleague who shipped the model's answer unread.

Where the job actually changed, and what to write there

The change is not that people use a tool. It is that a confident, plausible wrong answer now arrives inside otherwise good work, and somebody has to be the one who catches it. That is a judgment behavior, it sits on competencies most frameworks already carry, and it is the only part of AI use worth a rating at all.

So write those behaviors into the rows the framework already has:

• Judgment. Names which part of a deliverable came from a model and says why that part was safe to accept.
• Quality. Sends work back when the source behind a load-bearing claim cannot be produced.
• Ownership. Takes the consequence of an error that arrived through a tool instead of attributing it to the tool.
• Communication. Leaves enough of a trail that a reviewer can tell which decisions were made and which were generated.

Each of those is gradeable from artifacts a manager already reads. None of them asks anyone to declare how much AI they used, which is the disclosure a review should never require: it is unverifiable, it gets answered strategically the moment a rating depends on it, and it says nothing about the work. Whether those behaviors deserve a row of their own is a question for the framework, and the default answer is the rows you already have: should AI be its own competency, or folded into the ones you already have.

Write the behaviors as things a manager can point at

Every behavior in the review needs a place a manager could point to inside real work: a document, a pull request, a ticket thread, a decision memo. Assessment research pushes the same direction. Researchers building an AI literacy measure for a US Navy robotics programme reported that a scenario task simulating AI use on the job outperformed the abstract tests they had adopted from prior work or written themselves ⁴.

The rewrite is mechanical. Take the line item you were about to add, and ask what a manager would attach to it.

Before: *Uses AI tools effectively in day-to-day work. Rate 1 to 5.*

After: *Flagged the vendor-spend figure in the Q3 board deck as unsourced, traced it to a superseded contract, and the corrected number changed the recommendation. Demonstrated.*

The second version is longer, and that is the point. It is a claim with a receipt, so it survives a calibration meeting and a disagreement with the employee. The first version survives neither, and in calibration it collapses into whichever manager talks about AI most.

Two habits keep this honest. Collect the examples during the cycle instead of reconstructing them in review week, because reconstruction is where a tool-use anecdote quietly replaces an outcome. And treat course completion as attendance, never as evidence: a certificate says somebody sat through a session, and whether the behavior changed afterwards is a separate question with a separate answer (is AI training enough, or do you have to check the work afterwards).

What about the person who uses no AI at all?

Rate their work. If the output is on time, correct and at the level expected, there is nothing to mark down, and marking it down turns a review into a tool-adoption campaign. The question worth asking is narrower: is there a task where the team's standard method now runs through a model, and is this person's route producing a better result or only a slower one?

That question has a checkable answer, and it resolves one of three ways. The method is genuinely better and the person is behind, so coach it and note the gap the way you would note any other skill gap. The method is worse and they were right to refuse it, which belongs in the review as good judgment rather than as resistance. Or the difference does not show up in the output at all, in which case it is a preference and a review has nothing to say about it.

The reverse case deserves the same treatment. Somebody whose volume tripled this year has not automatically had a strong year, because volume is the easiest number on the page to raise without raising the quality behind it. What that does to the level above is its own question: what a promotion should be based on when AI produces a lot of the output.

See the benchmarks >