Interviewing
A ChatGPT-Only History Transfers. Your Stack Is the Ramp.
Most of a candidate's ChatGPT experience transfers to a Copilot and Claude stack. The durable half of AI skill is how someone decides what to hand over and how they check what comes back, and that decision is the same in every assistant. What does not transfer is your configuration (where the approval step sits, what the retrieval layer indexes, which actions need a human confirm) and the ordinary friction of a new interface. Price that as a ramp, not as a screening requirement.
The takeThe tool line in a posting is doing something other than what it says. It stands in for a bet that someone who has used the same product will need less help in month one, which is a real cost and a small one. The expensive failure, a confident wrong answer shipped without a check, is indifferent to which assistant produced it. Screen for the failure that costs money. Teach the setup.
Where Olive fits
Open a role and see what the work shows
An interview can hear a candidate describe the moment an assistant led them somewhere wrong; it cannot capture them noticing it. Olive puts that in front of the person as work: a role-grounded assignment, an assistant that will overreach if nobody stops it, and a human reviewer who writes or edits the findings on what happened with the timestamp attached.
Assess your shortlistWhat Does a ChatGPT-Only Candidate Already Carry Into Your Stack?
A candidate whose only assistant has been ChatGPT still carries the decision, which is the expensive half of AI skill. Every time that person used it, they decided what to hand over, how much context to give it, when to stop reading, and whether to check a claim against something outside the conversation.
None of that lives in a product. Judgment lives in the person, and judgment travels.
A pre-registered field experiment run in 2023 by Harvard Business School researchers put 758 Boston Consulting Group consultants through 18 realistic consulting tasks chosen to sit inside the model's capability. Consultants given GPT-4 completed 12.2% more tasks than the control group 1.
One task in the run sat outside the model's capability. There the consultants with no model at all came out ahead, by 19 percentage points against the two AI conditions combined: the control group answered correctly 84.5% of the time, the two AI conditions 60% and 70% 1.
Same tool, opposite direction.
The two AI groups differed in one thing, whether they had been handed an overview of prompt engineering, and the coached group scored the lower of the two 1.
Tool instruction did not rescue the judgment on that task. It moved with the wrong sign.
A mid-2023 model, one sitting, one task, and a frontier that has moved since 1. What survives those limits is a finding about people: many of the consultants handed a model could not tell which side of the line that task sat on.
Seeing that line is the durable skill, and no product teaches it. An AI skill is a decision, not a tool you have open, and which assistant produced the confident wrong answer matters far less than whether the person stopped and checked it.
Where Does Your Stack Actually Differ From a Chat Window?
What separates your stack from a chat window sits mostly in your building, not in the product documentation. A coding assistant that proposes a diff inside the editor asks for a different reading habit than a chat window returning prose. Past that, the parts a new hire has to learn are yours: which actions run without a confirm, what the retrieval layer has been pointed at, where the approval step sits, and what an administrator turned off.
Job analysis already separates a product from the kind of work it does. O*NET, the occupational database published by the US Department of Labor, ships a Technology Skills file whose stated purpose is to "Provide Technology Skills examples". Each of its 32,627 rows pairs one product name with the commodity class it is an example of.
Adobe Acrobat sits under document management software 2.
Two flags sit beside each row. O*NET marks an entry a hot technology when the requirement appears frequently across all employer job postings, and in demand when it appears frequently in postings for that particular occupation 2. A brand earns either flag because postings already carry it.
The class is the skill. The product is one instance of it.
An internal transfer learns the same local facts. Someone who has run your stack in another team still arrives with different permissions, a different index and a different reviewer, which is the tell that the ramp belongs to the deployment.
Put a date on that ramp, and start the clock only once the new hire has access to the tools.
Why Doesn't the Tool Name Predict Who Performs?
A tool name is unlikely to predict who performs, because experience measured at hire barely predicts later job performance, and a brand on a resume is the thinnest form of that experience. A 2019 meta-analysis of 81 independent samples put the corrected correlation between prehire work experience and later job performance at .06, and experience with relevant tasks, jobs or occupations did no better, at about .07 3.
The measure a resume screen reads is the measure that failed.
The meta-analysis is careful about scope. The finding covers experience counted at hire, not tenure inside the job, which is a different and more favourable literature. Prehire experience did predict a little better when a worker first starts a new job, and task-level experience did predict training performance, though both exceptions rest on small subsets of studies 3.
Relevance did carry signal in a different kind of measure, a test of job knowledge. In the job knowledge meta-analysis Sackett and colleagues re-examined in 2022, all 164 studies together produced a mean observed validity of .22, while the 59 studies using knowledge tests built for the job in question produced .31 4.
Both figures come from subsets of one meta-analysis, with no controlled trial behind either, and a job knowledge test assumes candidates who already hold the knowledge 4.
Take the direction and leave the decimal: within one test format, what the test was about moved the result.
One vendor credential publishes how much of its weight sits on a single product. Microsoft's Azure AI Fundamentals exam, AI-901, updated 15 April 2026, puts 40-45% of its weight on identifying AI concepts and capabilities. Another 55-60% sits on implementing solutions in Microsoft Foundry, and the exam page also asks for knowledge of Python coding syntax 5.
Google Cloud's Generative AI Leader certification, open to anyone in any job role with or without hands-on technical experience, runs 90 minutes, costs $99, requires no prerequisites, and expires after three years 6.
Neither exam page publishes a pass rate, a holder count, or any link to job performance, so neither says anything about the people holding the credential 56.
A tool line on a resume has the same problem, and a posting that asks for one inherits it. The replacement worked out for year counts, what to require instead of years of AI experience, asks for no tool name either.
Ask What the Assistant Did That They Did Not Expect
One question reads the half of the skill that transfers, and it fits inside a screening call. Ask a candidate about a time the assistant did something they did not expect, then ask what they changed about their own process afterwards. Transferable skill shows up as a changed process. Tool familiarity shows up as a changed setting, and both answers are worth hearing once you can tell them apart.
A changed-process answer sounds concrete. They stopped asking for a finished document and started asking for three options first, because the finished version hid the choice. Or they began checking the model's figures against the source before quoting them, because one number survived three drafts and was wrong.
A changed-setting answer sounds like a release note. They turned a suggestion mode off, moved to a longer context window, wrote a system prompt. Useful, and it expires with the next version.
The weakest answer is a general one, that they check the output, and it names no claim at all. So the follow-up carries the round: testing the answer that says they always double-check means asking which claim they checked, against what, and what changed as a result.
One proprietary agent framework and a chain of internal tools is where this fails. Product knowledge is then a far larger share of the work, the ramp runs longer, and a small team may not be able to carry it.
Write the Requirement as a Class, Not a Brand
Name what the assistant has to do, and let the brand be an example of it. A posting that asks for experience with an assistant that proposes changes inside a codebase, or with a retrieval-backed assistant over internal documents, describes work.
A posting that names two products describes a purchase order, and it screens out the candidate who did the same work somewhere else.
Then write the line that lets equivalent work in. A posting accepting comparable work on another assistant reaches the person who did the identical job under a different vendor contract.
Price the ramp into the onboarding plan. Name one reviewer, one review period, and the two documents nobody reads until they need them: the permission list, and whatever the retrieval layer has indexed.
Say in the offer conversation which parts of the first month are learning the setup, because a candidate who hears that also hears what will be judged and what will not.
A ChatGPT-only history does not mean the candidate is missing the skill the role is for.
They are missing your approval step, your index and your review surface, and every one of those is written down somewhere in your own building. Hand them that document on the first day. Then watch the half no document teaches.
Common questions
Does ChatGPT experience count toward a role that runs on Copilot and Claude?
Most of a ChatGPT-only history counts. The part that matters is how a person decides what to delegate and how they check what comes back, and that decision is the same in every assistant. What a different product changes is local: where the approval step sits, what the retrieval layer indexes, which actions need a confirm, and the ordinary friction of a new interface. Treat those as a ramp with a named review period, and screen instead for one story about a time the assistant did something unexpected and what the candidate changed afterwards.
Should a job posting name the specific AI tools the team uses?
Name the tools as examples, not as the requirement. Describe what the assistant has to do in the role, an assistant that proposes changes inside a codebase, or one that answers over internal documents, then list the products as instances and accept comparable work on another. A brand-name requirement screens out people who did the identical job under a different vendor contract. O*NET, the US Department of Labor's occupational database, records product names exactly that way: as examples classified under a broader commodity class.
How long does it take to become useful on a different assistant?
No public measurement isolates the ramp to a new assistant, so a number here would be invented. What can be decided is the plan: name a review period, name who reads the person's work during it, and hand over the two things no product documentation covers, the permission list and whatever the retrieval layer has indexed. Judge the ramp on whether the person's checking habits hold under an unfamiliar interface rather than on speed. And do not start the clock until access exists, because a hire waiting on an account is not ramping.
Do vendor AI certifications prove the skill transfers?
Two vendor exam pages point the other way. Microsoft's Azure AI Fundamentals exam, AI-901, puts 55-60% of its weight on implementing solutions in Microsoft Foundry and asks for knowledge of Python coding syntax. Google Cloud's Generative AI Leader certification is open to anyone in any job role, runs 90 minutes, costs $99, and expires after three years. Both pages describe what an exam asks, and neither publishes a pass rate, a holder count, or any link to job performance. A pass evidences familiarity with one vendor's vocabulary and product line, which is the half that decays.
What if the candidate has only used the free consumer version?
The judgment still transfers, and one thing genuinely needs checking. A consumer account has no admin controls, no employer retention policy, and no indexed internal corpus, so a candidate may never have faced the question of what may be pasted where. Ask a case: hand them a document carrying client identifiers and ask what they would put into the assistant and what they would keep out. That answer is about the role, not about which product they arrived from.
References
- 1. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality (Working Paper 24-013) ✓ mitsloan.mit.edu Supports the pre-registered experiment with 758 Boston Consulting Group consultants across 18 tasks inside the model's capability and the 12.2% more tasks completed; the single out-of-frontier task where AI users were 19 percentage points less likely to be correct, 84.5% of the control group against 60% and 70% in the two AI conditions; and the lower score in the condition given an overview of prompt engineering. Cited for one task on a mid-2023 model, never as a general claim about assistants.
- 2. Technology Skills - O*NET 29.1 Data Dictionary ✓ onetcenter.org Primary source opened for this article. The sentences relied on are the file's stated purpose, 'Provide Technology Skills examples'; 'The columns Commodity Code and Commodity Title classify the example under the United Nations Standard Products and Services Code (UNSPSC).'; 'The Hot Technology column indicates requirements frequently included across all employer job postings.'; 'The In Demand column indicates requirements frequently included in employer job postings for the particular occupation.'; and 'There are a total of 32,627 rows of data in this file.' The Adobe Acrobat entry filed under document management software is from the page's own Data Example table.
- 3. A meta-analysis of the criterion-related validity of prehire work experience ✓ digitalcommons.unf.edu Supports the 81 independent samples, the .06 corrected correlation between prehire work experience and later job performance, the roughly .07 for experience with relevant tasks, jobs or occupations, and the authors' two named exceptions, both from small subsets of primary studies: prehire experience is somewhat more predictive of job performance when workers first start a new job, and measures of task-level experience predict training performance.
- 4. Revisiting Meta-Analytic Estimates of Validity in Personnel Selection: Addressing Systematic Overcorrection for Restriction of Range ✓ static1.squarespace.com Supports the job knowledge comparison: 164 studies with a mean observed validity of .22 against the 59 studies using job-specific knowledge tests at .31. Cited for direction only, since it compares subsets of one meta-analysis and presupposes candidates who already hold the knowledge.
- 5. Exam AI-901: Microsoft Azure AI Fundamentals ✓ learn.microsoft.com Supports the 15 April 2026 update date, the 40-45% weighting on identifying AI concepts and capabilities against 55-60% on implementing AI solutions using Microsoft Foundry, the stated requirement of knowledge of Python coding syntax, and the absence of any published pass rate or holder count.
- 6. Generative AI Leader certification ✓ cloud.google.com Supports the certification being open to anyone in any job role with or without hands-on technical experience, the 90-minute format, the $99 fee, the absence of prerequisites, the three-year validity, and the absence of any published pass rate or holder count.
6 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.