AI Skill Doesn't Survive the Resume Stage

What Can a Resume Tell You About AI Skill?

Which claims a candidate is prepared to make, and nothing about whether they hold up. A tool list, a certificate line and a project bullet are assertions, and each one is now cheaper to produce than to check. Read them the way you read a reference to a former employer: a place to point a question later, not a finding you can act on.

A claim with a task inside it ("rebuilt the weekly variance commentary with a model and now reviews it instead of writing it") names something a later stage can test. An inventory ("ChatGPT, Claude, Copilot, Midjourney, Notion AI") names nothing, because the same five words appear on most of the pile.

So the resume stage is doing routing work rather than evaluation work. It tells you the occupation, the seniority, whether the previous job was in AI-exposed work, and which one sentence is worth returning to. That is a useful stage. It is not a screen for skill, and asking it to be one is where the funnel breaks. When a candidate lists six AI tools the useful move is choosing which of the six to ask about, not counting them.

Two claims are worth marking on almost any resume for an AI-exposed role. The first is the task the candidate says moved, because that is where a follow-up question has purchase. The second is any number attached to it, since a number invites the question of how it was measured and who checked. Mark them and move on. The stage has done its job in under two minutes.

Why Certificates and Tool Lists Measure Exposure

Because exposure is what they are built to measure, and the exam guides say so plainly. The AWS Certified AI Practitioner names a target candidate with up to six months of exposure who uses but does not necessarily build AI solutions, and lists developing or coding models as out of scope ¹. It is a recognition test under time pressure, with no performance task anywhere in it.

The other credential that shows up constantly is stranger still. Microsoft's Azure AI Fundamentals exam page asks for knowledge of Python syntax and programming techniques and familiarity with Azure resources, and the majority of the exam weight sits on implementing solutions inside one vendor's platform ². A non-technical manager reading "AI Fundamentals" on a resume is not reading what they think they are reading.

Neither observation is an argument against certificates. Somebody who passed one sat down and learned a body of material, which is worth something. It is an argument against treating the line as evidence of judgment, because the exam did not ask for judgment and does not claim to have measured it. Whether a prompt engineering certificate beats a work sample is the sharper version of the same question.

Stop Pre-Filtering on the Claim

A pre-filter on an unverifiable claim does not remove weak candidates. It removes the ones who described themselves less fluently. Self-reported and tested AI skill barely move together: when 288 teachers took both a self-report and a knowledge test built on the same four-part framework, correlations between the two ran from 0.07 to 0.24 ³. Sorting on the self-report sorts on writing.

That study is teachers in Taiwan, not job candidates, so the rate in it belongs to that cohort and stays there. What travels is the shape of the finding: the two instruments were built on one framework and still did not agree, and the profiles included people who underrated themselves as well as people who overrated themselves. A pre-filter loses both groups in the same pass.

The replacement is cheap. Set the requirement at the level of the work rather than the claim, let everyone who clears the occupational bar through to the same short task, and spend the screening effort on reading the task instead of ranking the paragraphs. Going straight from application to work sample is the fuller version of this move, and it turns on arithmetic: applicant count times review minutes against the hours the requisition actually has.

Test the Behavior the Claim Points At

The behavior worth testing is what a person does when the assistant is confidently wrong. In the BCG field experiment, on a task deliberately chosen to sit outside the model's competence, consultants using GPT-4 were 19 percentage points less likely to reach the correct answer than the control group ⁴. Nobody in that condition could see which side of the line their task was on.

One sentence covers the skill, and it decomposes into moments you can watch: framing the problem before generating anything, asking for a source on the claim that matters, keeping the judgment that should not be handed over, throwing away output that looks finished, and checking a result against something outside the conversation. Each of those is visible in a forty-minute task and invisible on any document.

Researchers building an AI literacy assessment for a US Navy robotics programme reached the same design conclusion: a scenario task simulating AI use on the job outperformed the knowledge tests they had adopted or written, and prevailing assessments over-weight technical foundations against practical work ⁵. That is one programme's comparison rather than a published margin, so treat it as a design argument.

Run the task after a human has expressed interest rather than in front of an open posting, keep it inside an hour, and write the answer key first. Verifying an AI claim in ten minutes with no assessment budget is the low-cost floor if that is all the process can carry this quarter, and the two questions worth asking in a phone screen decide which claim the task should aim at.

A usable task looks ordinary. Hand the candidate a real artifact from the job with two defects planted in it, an assistant that will happily agree with whatever they propose, and a deliverable a colleague would actually receive. Then read for what they refused. The candidate who ships the assistant's second draft and the candidate who threw out the first three are both visible within ten minutes of reading.

See a sample report >