How Do You Find Out What AI Does in a Role Before Writing the Job Post?

How do you find out what AI actually does in a role?

Pull the occupation's published task list, mark the tasks a model can already draft a plausible first pass on, then spend twenty minutes each with three people who do the job now. The list gives you tasks nobody in your company wrote. The calls tell you which of those tasks the work actually contains this quarter, and where the model's output goes wrong.

The job description sitting in your ATS is the wrong starting point. It was written for the last hire, edited by two people with different roles in mind, and it describes duties rather than tasks. A duty is "own the reporting pipeline." A task is "clean and manipulate raw data using statistical software," small enough that you can say whether a model does it today.

Two passes, in this order:

• The external pass. One occupation, one published task list, about an hour. This is the pass that hands you tasks nobody at your company has an opinion about.
• The practitioner pass. Three twenty-minute calls with people doing the work now. This is the pass that says which of those tasks is real this quarter and what the model gets wrong on them.

Do them in that order, or the calls turn into a wish list. Handed a blank page, a practitioner describes the job they wish they had; handed sixteen task statements, they cross off four and tell you why. If you have not settled which roles need AI skills at all, run that triage first. This is half a day per role, and it is wasted on a role that does not need the requirement.

Start with the occupation's task list, not the job description

O*NET publishes task statements for every SOC code, and they are specific enough to mark. Data Scientists (15-2051.00) carries sixteen of them, from "clean and manipulate raw data using statistical software" to "compare models using statistical performance metrics" ². Read each one and mark it: a model drafts this end to end, a model helps, or a model cannot touch it.

Three marks, not five. A four-point scale invites hedging, and the hedge is where this exercise dies.

• Drafts it end to end. The output would be shippable if nobody checked. Most writing, summarizing and first-pass code sits here.
• Helps. The model gets you to a middle state faster, but somebody finishes it with information the model does not have.
• Cannot touch it. The task needs an act in the world: a call, a system the model cannot reach, a judgment about a person.

The same occupation page carries a technology skills section naming the software the occupation actually uses ², which is a better source for the tools line in your post than whatever was trending on a vendor blog last month.

Mind the clock on that source. That task list moves on a published schedule (updated quarterly, with the primary annual update in the third quarter ¹), so it lags a fast-moving field by months rather than by years. The lag is exactly what the practitioner calls are for. It is not a reason to skip the list: a slightly stale external list still beats a fresh internal opinion, because nobody who wants the requisition approved wrote it.

What do three practitioner calls add that the task list can't?

Three things the list cannot know: which of those tasks your version of the job actually performs, what the model reliably gets wrong on them in this field, and what a good practitioner refuses to hand over. Twenty minutes each, three people, at least one from outside your company. Ask about last week, not about AI in general.

Five questions, in this order. Each is about a specific recent instance, because "how do you use AI?" returns a description of a workflow nobody has.

1. Show me the last thing you handed to a model. What went in, what came back, what you did with it. 2. What did you keep for yourself, and why? The refusal is the signal. A practitioner who says "I write the recommendation myself, because the model picks whichever option the sources describe most confidently" has just handed you the requirement. 3. When did it last give you something plausible and wrong? Field-specific failure modes are the most valuable thing on the call: the fabricated citation, the ratio computed off the wrong denominator, the clause that does not exist in this jurisdiction. 4. How would you have known? This is the question the job post is really about. A usable answer names a source, a recomputation, or a person. 5. What would a new hire get wrong in month one? Practitioners answer this precisely, and it maps straight onto the interview loop.

Take notes as quotes rather than as a summary. A sentence in a practitioner's own words survives the argument with the hiring manager; your paraphrase of it does not.

Three is the floor, and the third call is the one that earns its time, because two people from the same team give you one opinion twice. Get one from outside if you can: a former colleague, a candidate you did not hire, someone in the same function at a company one size up. If the three disagree about what the model is used for, the disagreement is the finding, and the post should ask for the judgment rather than the tool. What AI fluency means on a job description is downstream of these calls, not upstream of them.

Why does the answer change by occupation and by quarter?

Because AI use tracks information work, and every occupation holds a different amount of it. A Microsoft Research team classified 200,000 anonymized Copilot conversations against occupational work activities and found the most common and most successful assisted activities are creating, processing and communicating information ⁴. That is why a data analyst and a product manager, both nominally AI-heavy roles, have different exposed tasks.

Depth varies more than breadth. Anthropic's usage index found roughly 36% of jobs showed AI use across at least a quarter of their tasks, and only about 4% across at least three-quarters ⁵. The honest shape of almost every role is a handful of exposed tasks inside a job that is mostly not exposed, which is why a post demanding general AI proficiency screens on something the work barely exercises.

The quarter matters as much as the occupation, for three reasons:

• The published list lags. It moves on a release schedule ¹, and what a model can draft moved between releases.
• The tools churn. A named tool in a post written in March can be the wrong tool by September, and a candidate who used the competing product is filtered out for nothing.
• What practitioners refuse to hand over moves too. A task worth doing by hand last year is delegated with a check this year, and the check is the skill.

The practical rule: rebuild the inventory when you open the requisition, not when you happen to remember. A six-month-old inventory is a reasonable input. A two-year-old one describes a job that no longer exists, which is the same failure as screening on keywords every resume now matches.

Turn the inventory into one line in the job post

Write one requirement, drawn from the tasks that survived both passes, naming the act rather than the tool. "Checks a model's output against the source before it goes in the deck" is assessable. "Proficient with AI tools" is not, and it attracts everyone. Keep the inventory itself. It is the record that makes the requirement defensible later.

That record matters more than most founders expect. Under the Uniform Guidelines on Employee Selection Procedures, a selection procedure justified on content grounds rests on a job analysis that "should focus on the work behavior(s) and the tasks associated with them," and the behaviors chosen for measurement "should be critical work behavior(s) and/or important work behavior(s) constituting most of the job" ³. A dated task inventory citing its sources and three named practitioners is that document. A requirement someone added because it sounded current is not.

Three moves from the inventory to the post:

1. Cut the requirement down to the tasks marked "drafts it end to end" that also cost real money when they come back wrong. Everything else is a tool the hire can pick up in their first week. 2. Write the act, not the noun. You already have the sentence from the calls: a practitioner said it about last Tuesday. Use theirs. 3. Decide whether it is a requirement or a preference. Requiring AI experience outright narrows the pool in a specific direction; a preference costs nothing and still tells applicants what the work is.

Then write it so a lawyer and a candidate read it the same way. An AI-skills requirement that is not legally vague is a shorter sentence than most people write, and the round that checks it should exist before the post goes up. If the loop has nothing that tests the requirement, you have written a filter you cannot defend and a promise you cannot keep.

See a sample report >