Build the AI Question From the Role's Own Tasks in an Afternoon

Why doesn't a generic AI question list work?

Because the list copies the wording and drops the derivation. Every per-function version runs the same handful of questions with the job title changed: describe a time you used AI, what are its limits, how do you check the output. Those have well-known good answers now, so a prepared candidate returns one and you learn what they read last week. A question built from a task only your team performs has no rehearsed answer available.

The practical tell is whether you can say, before the interview, what a good answer contains. Ask a nurse how they use AI at work and forty answers are defensible, which means none of them separates anyone. Ask what they would do when the drafted discharge summary lists a medication the patient stopped two admissions ago, and there is a right answer, a wrong answer, and a small band of partly right answers that show you exactly where the person sits.

The derivation transfers where a copied list does not, and that is the whole difference. Before you write anything, it helps to know what AI actually does in this role day to day, because the answer usually differs from what the job description claims and from what the person who wrote it assumed.

Start from the three tasks the role does most

Open the job description, ignore the responsibilities list, and write down the three things this person will genuinely spend most of a week doing. Then ask the two people already doing that job which step of each task now begins with something a machine produced. The result is short, specific to your team, and the only input the rest of this needs. It usually takes under an hour of conversation.

What comes back will be narrower than the job title suggests. Indeed's GenAI Skill Transformation Index rated almost 2,900 work skills found in US job postings and mapped them onto more than 53.5 million postings, placing 40% of skills in minimal transformation, 19% in assisted, 40% in hybrid and 1% in full, with 46% of the skills in a typical posting falling into the hybrid or full categories ¹. Read that carefully before repeating it: large language models produced those ratings by scoring skills, and nobody watched the work being done, so it measures what current models are judged capable of on one job board over twelve months. Hybrid in that scheme explicitly means human oversight is still required.

So the interesting unit is the seam inside a task, not the task itself. A financial analyst still builds the model; the comparable-company pull now arrives pre-drafted. A recruiter still writes the requirement; the first version comes out of a tool. A paralegal still files; the summary of the deposition arrived already written. Write down the step, the artifact it produces, and who receives that artifact next. Three tasks, three seams, three receivers, one page.

Write the question at the seam where a bad draft escapes

Pick the point in each task where an unchecked draft leaves your team: sent to a client, filed with a payer, merged, published, read aloud to a patient. Put the question there. That is where the cost lands, so it is where a candidate's habits become visible and where you can say in advance what a passing answer has to contain. Anything upstream of the seam is preference, and preference has no right answer.

Aim there because people are poor judges of which side of the line a task sits on. In the Boston Consulting Group field experiment, on one task deliberately chosen to sit outside the model's capability, consultants using GPT-4 were 19 percentage points less likely to reach the correct answer: 84.5% of the control group got it right against 60% and 70% in the two AI conditions ². One task, one sample, a model from 2023, and the group given a prompt-engineering overview did worse than the group given none. What travels is the failure shape, which is a confident wrong answer on work that looked routine.

Two question forms cover most roles once you have the seam:

1. The artifact question. Hand over a real first draft the assistant produced for that task, with one thing wrong in it that your field would catch. Ask what they would not send, and how they would check the part they are unsure about. Fifteen minutes, and it is the closest thing to the job an interview can hold. 2. The boundary question. Ask which part of this task they would not hand to an assistant at all, and what it would cost if they did. The answer turns on accountability, and it is hard to fake without knowing the work.

Grade both against what the artifact actually contained rather than against how the answer sounded. The four lines worth putting on the scorecard covers what to write down while they talk.

What does this look like in a non-technical role?

The derivation does not change; only the artifact does. A demand generation manager checks a claim inside a drafted campaign brief. An accounts payable clerk checks a coded invoice against the contract terms. A charge nurse checks a drafted handover against the chart. In every case the moves are identical: name the seam, produce the artifact, ask what they would not send onward and why.

Most job titles that mention AI now sit outside technology. Indeed Hiring Lab counted job titles carrying AI language in the employer's own raw title and found US AI-touched titles rising from 264 in the first quarter of 2022 to 822 in the first quarter of 2026, with 63% of them outside tech occupations ³. That counts distinct titles with at least five postings in a quarter, so it tracks how far the vocabulary has spread across role names and says nothing about volume of demand, and the count dipped before it rose. The direction is enough for the point here: the manager writing an AI question this month is often not a technical manager.

Three worked seams, to show how little the shape changes:

• Demand generation. Seam: the brief goes to an agency. Question: here is the drafted brief, one of these three claims about the product is not true, tell me which and how you would confirm it.
• Accounts payable. Seam: the coded invoice enters the payment run. Question: the assistant coded this to the wrong cost center and the total matches the PO. What would you check before approving it?
• Ward handover. Seam: the summary is read at shift change. Question: this drafted handover omits an allergy that is in the chart. Walk me through what you do in the next two minutes.

Each one has a right answer, each takes under ten minutes, and none of them appears on a list of interview questions for that job title. If the role sits outside engineering and this still feels like a technical exercise, screening for AI judgment in finance and marketing roles works the same derivation through longer examples.

See how it works >