Policy
Your AI Requirement Screens Before the First Application Arrives
An AI requirement in a job posting cannot be measured with a selection rate, because the people it turns away never enter the applicant records that rate is computed from. Two instruments work before anyone applies. Test whether the task the requirement stands for actually needs it, by checking who can do that task without the stated experience. Then run alternate wordings of the same posting and compare who applies to each.
The takeThe reporting gap is not a defect in anyone's dashboard. Fairness measurement in hiring begins at the applicant pool because that is where the records begin, and a posting requirement has finished its work one step earlier. A team that reports a clean impact ratio at every gate has reported honestly about everything it can see and said nothing about the line that filtered hardest. A requirement line filters before anything else does, yet the assessments after it are the part that gets validated. The order of that scrutiny is backwards.
Where Olive fits
Open a role and see what the work shows
A requirement in a posting asks for prior exposure; a role-grounded session asks for the work. Olive runs the second: a 40-to-60-minute occupational assignment with an AI assistant, returned as six findings that a person writes or edits, each anchored to a timestamped excerpt, with the candidate granted the same report.
Assess your shortlistWhy can't a selection rate reach someone who closed the tab?
Because the arithmetic needs people who applied, and the requirement removed them first. A selection rate is the share of one group that clears a gate, and an impact ratio divides it by the highest group's rate at that same gate. Someone who read the posting, saw a line demanding two years of AI experience, and closed the tab entered no gate at all.
No row exists in the system to divide.
Every gate after the posting leaves a record, which is why the denominator at each gate is who entered that gate works as a rule from the resume screen onward. The posting is the one gate whose rejections are unwritten.
The problem is old. Written in 1978, the four-fifths paragraph of the Uniform Guidelines, 29 CFR 1607.4(D), says a selection rate below 80% of the highest group's rate will generally be regarded by federal enforcement agencies as evidence of adverse impact.
Even where every applicant is on file, the four-fifths rule is a trigger for scrutiny rather than the legal line. A ratio you cannot compute at all is a weaker starting point still.
The same paragraph reaches smaller differences too, including where a user's actions have "discouraged applicants disproportionately on grounds of race, sex, or ethnic group" 1.
The regulation names discouragement and supplies no way to count it.
Would the people already doing the job clear your AI line?
Name the task the requirement stands in for, then check whether people without the stated experience can do it. An AI requirement is rarely about AI. It stands for something concrete: drafting from a model and catching what the model got wrong, or holding a judgment the tool would take if nobody stopped it.
Write that down and the line becomes checkable.
Run the check on people you already employ.
Stated requirements drift away from the people doing the work, and the drift has been measured. Joseph Fuller and Manjari Raman analyzed more than 26 million job postings for Dismissed by Degrees, published in 2017 by Harvard Business School with Accenture and Grads of Life. In 2015, 67% of production supervisor postings asked for a college degree while 16% of employed production supervisors held one 2.
One occupation, chosen to make the point: posting text on one side, incumbent workers on the other, both from 2015.
An AI line can drift the same way, and nobody holds a baseline for what the role needed last year.
The comparison costs an afternoon. Take the four or five people currently doing the job well, and count how many would clear the line as written. If most of them would not, the requirement is describing a candidate somebody imagined rather than the work.
The answer arrives before you post, and it needs no pool at all.
Whether the job post should require AI experience at all is the decision one step earlier, and the usual answer there is to name the act rather than the credential.
Post the line two ways and watch what each version converts
Post the same role twice with different wording, split the traffic between the versions, and compare what each one converts. The people who read the posting are the only group upstream of your applicant table you can watch move. The comparison never identifies who stayed away or why, but it does tell you what one version of the line costs in readers.
Part of the loss already sits in your ad reporting, in clicks that never became applications.
Appcast's 2025 Recruitment Marketing Benchmark Report covers more than 281 million clicks and 25.6 million completed applications, drawn from over 1,300 US employers across 2024.
The median apply rate on paid long-apply job ads ended that year at 6.1% 3. More than nine in ten clicks did not become an application. Apply rate measures the form and the ad targeting as much as the words, so the figure is a baseline rather than a verdict on any single line.
Posting language does move how appealing a job reads, and the measured mechanism is not the one people assume.
In Study 5 of a 2011 paper in the Journal of Personality and Social Psychology, Gaucher, Friesen and Kay gave 118 female undergraduates one version of a single real estate agent advertisement. Those who read the masculine-worded version rated the job less appealing, 3.86 against 4.34 on a seven-point scale, and reported less anticipated belonging 4.
Their rating of their own skill for the job did not move. The authors added those items to rule out a simpler story: that the wording made women feel less able. They found nothing.
Nobody in that study applied to anything. Undergraduates rated a hypothetical advertisement online for course credit, and the gap is about half a point in one occupation.
The carry-over to an AI line is an inference, and worth naming as one. The experiment supports something narrower: a phrase can change how a reader judges their own fit, through belonging rather than through doubt about competence.
If the alternate version needs something to say in place of the requirement, what a posting should accept instead of on-the-job AI experience is the wording problem underneath it.
How common is on-the-job AI use where you hire?
Look up how common the experience is in the labor market before the line goes into the posting. A requirement's reach depends on how many people hold the thing it asks for. If on-the-job AI use is still a minority behavior where you hire, then one sentence removes most of the market, whoever those people turn out to be.
A usable baseline comes from a nationally representative US survey.
The Real-Time Population Survey ran two waves in August and November 2024, with 5,014 and 5,329 responses. Of employed respondents, 23% had used generative AI for work at least once in the previous week, and 9% used it every work day 5.
Both figures are self-reported, and using a tool once in a week is a low bar. Adoption numbers of this kind age in months, so date the one you write into the requisition.
A requirement written against a base rate near a quarter of employed respondents reaches further than the same words will once that rate rises.
Neither that 23% nor its successor tells you which groups sit on the far side of the line. Looking the number up is still worth ten minutes, because the requisition meeting then stops arguing about whether AI experience is common.
Sizing the line is also the first move toward finding who it lands on. An AI requirement can create age impact, and the four-fifths paragraph will not find it: the text speaks to race, sex and ethnic groups only 1.
Four things on file, and how little a rewrite moves hiring
Record the wording, the dates it ran, the traffic it drew and the apply rate it produced, for every version you post. A posting experiment with no record is an opinion two quarters later. The file is also the only proof the requirement was reconsidered at all, and it outlasts the result of any single comparison.
Suppose the words in a posting barely move who ends up hired. Then every instrument above is measuring a small thing carefully.
The closest measured case is about degrees rather than AI. At large firms that had dropped a degree requirement, the Burning Glass Institute and Harvard Business School matched 11,332 roles against the career histories of roughly 65 million US workers.
Inside those roles, the share of hires without a bachelor's degree rose about 3.5 percentage points. Only 3.6% of roles dropped anything, so the net effect across the market was 0.14 percentage points, about 97,000 workers a year out of 77 million hires 6.
An employer that rewrote a requirement and then hired the same people has learned something real about itself, though not the thing it set out to measure.
A person who reads a posting and moves on leaves nothing behind for a system to count.
So hold what a posting experiment can hold: the exact words, the dates each version ran, and how many readers each one converted. Somebody will eventually ask what the AI line cost. A dated file answers that better than a recollection.
Common questions
Can you compute an adverse impact ratio on people who never applied?
No. A selection rate is a share of the people who entered a stage, so anyone removed by a line in the posting is missing from both the numerator and the denominator. The four-fifths paragraph of the Uniform Guidelines does mention applicants that a user's actions have discouraged disproportionately, but it names no method for identifying or counting them. Applicant-flow arithmetic starts one gate too late for a posting requirement, and no amount of care with the data fixes that.
What is the cheapest way to check an AI requirement against the job?
Count how many people currently doing the job well would clear the line as written. Name the task the requirement stands for first, in the occupation's own words, then check whether the stated experience is what that task needs. If most of your strong performers would fail the line, it describes an imagined candidate rather than the role. The whole check runs on people you already employ, takes an afternoon, and needs no applicants at all.
How do you A/B test a job posting?
Post the same role in two wordings, split paid traffic between them, run both over the same dates until each has drawn enough clicks for a gap to outlast chance, and compare apply rates. Keep everything except the requirement line identical, including the title, the salary range and the application form. What the comparison measures is the reading population: who clicked and then finished. It cannot tell you which groups declined, so treat a gap as a reason to look at the wording again rather than as a demographic finding.
Is a low apply rate proof that the AI requirement is the problem?
No. Apply rate is completed applications divided by clicks, and it moves with the application form, the ad targeting and the state of the labor market as much as with any sentence in the posting. Appcast's 2025 benchmark report, drawn from over 281 million clicks at more than 1,300 US employers in 2024, put the median on paid long-apply ads at 6.1%, so a low figure is ordinary rather than diagnostic. Only a comparison between two versions of your own posting isolates the wording.
Does changing a requirement in the posting change who gets hired?
Often less than expected. Matching 11,332 roles at large firms that dropped a degree requirement against roughly 65 million US career histories, the Burning Glass Institute and Harvard Business School found the share of hires without a degree rose about 3.5 percentage points inside those roles, while only 3.6% of roles dropped a requirement, leaving a net market effect of 0.14 percentage points. Requirement text is a filter on who reads and applies. It is not on its own a hiring practice.
What should the record hold if somebody asks about the AI line later?
Four things: the exact wording of every version posted, the dates each ran, the traffic each drew, and the apply rate each produced. Add one line naming the task the requirement stands for and how you checked that current job-holders meet it. That file shows the line was reconsidered rather than copied forward. It does not prove the requirement was fair, because nothing computed from applicants can, but it is the strongest record available before an application exists.
References
- 1. 29 CFR 1607.4 - Information on impact (Uniform Guidelines on Employee Selection Procedures, 1978) ✓ law.cornell.edu Supports the four-fifths paragraph's text, its reference to selection rates by race, sex or ethnic group, and its clause extending adverse impact to cases where a user's actions have discouraged applicants disproportionately.
- 2. Dismissed by Degrees: How Degree Inflation Is Undermining U.S. Competitiveness and Hurting America's Middle Class ✓ gradsoflife.org Supports the claim that stated posting requirements drift above what incumbents hold: 26 million postings analyzed, and in 2015, 67% of production supervisor postings asked for a degree against 16% of employed production supervisors holding one.
- 3. 2025 Recruitment Marketing Benchmark Report, U.S. Edition ✓ info.appcast.io Supports the apply-rate baseline used as the measurable top-of-funnel loss: over 281 million clicks and 25.6 million applies from more than 1,300 US employers in 2024, with the median long-apply rate ending the year at 6.1%.
- 4. Evidence That Gendered Wording in Job Advertisements Exists and Sustains Gender Inequality ✓ ideas.wharton.upenn.edu Supports the claim that posting wording moves appeal and anticipated belonging without moving perceived personal skill: Study 5, 118 female undergraduates, 3.86 against 4.34 on appeal, with the personal-skill items showing no effect.
- 5. The Rapid Adoption of Generative AI (NBER Working Paper 32966) ✓ nber.org Supports the external base rate for on-the-job AI use: in the August and November 2024 Real-Time Population Survey waves, 23% of employed respondents had used generative AI for work in the previous week and 9% used it every work day.
- 6. Skills-Based Hiring: The Long Road from Pronouncements to Practice ✓ static1.squarespace.com Supports the complication that editing requirement text moves hiring very little: 11,332 roles matched against roughly 65 million career histories, a 3.5-point within-role shift, only 3.6% of roles changing, and a 0.14-point net effect of about 97,000 workers out of 77 million hires.
6 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance, not legal advice. Hiring rules differ by state and country and change often; check anything here against your own counsel before you act on it.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.