A Bias Audit Scores the Tool You Bought, Not the Way You Use It

What does a bias audit actually compute?

Selection rates and impact ratios by category, and nothing else. Under the New York City rules that define it, a bias audit must compute a selection rate and an impact ratio for each category, separately for sex, for race or ethnicity, and for intersectional categories of sex, ethnicity and race ¹. It is group arithmetic on a data set, on a date.

Three limits sit inside that definition and all three point the same way. An independent auditor may exclude a category representing less than 2% of the audit data from the impact-ratio calculation, which most often drops the smallest groups in the pool, the ones a bias audit exists to protect. An employer that has never used the tool may rely on an audit built entirely on other employers' historical data, and any employer may rely on one built on test data where there is too little real data for a statistically significant audit ¹. And nothing anywhere in the arithmetic touches accuracy or job-relatedness.

The law that produced this shape is local, and that matters when the summary gets read as a general certificate. In force since January 2023, the New York City rule bars an employer from using an automated employment decision tool on a New York City candidate unless a bias audit was performed within the prior year, a summary of the results is posted publicly, and the candidate received notice at least 10 business days before the tool is used ². It is a disclosure-and-audit regime rather than a ban: a tool that does badly in its own audit is still lawful to use there, provided the audit is posted.

What does the audit not answer?

It does not say whether the tool predicts anything about performance in this job, whether it works for someone using a screen reader or asking for extra time, or what happens to candidates in a category it could not measure. Those are the three questions a buyer actually has. The last one covers every category the rule never asked the auditor to look at.

Disability is the clearest of the three, and the agency said so plainly while its guidance was still posted. A vendor's bias-free claim usually means steps taken against Title VII adverse impact on race, sex, national origin, color or religion, and those steps are typically distinct from what disability requires: because each disability is unique, an individual can still be screened out regardless of how well other people with disabilities fare on the same assessment ³. That document came off the agency's website in January 2025 and never had the force of law, so it belongs in a memo as what the EEOC said in 2022. The statute underneath it was not touched.

The prediction question is separate evidence entirely, and the two labels get run together constantly. The difference between a validated assessment and a bias-audited one is the short version; whether a vendor's study covers your roles at all is a different check with conditions of its own.

In the first publicly documented cooperative audit of a commercial candidate-screening vendor, the auditors found that pymetrics did faithfully implement its stated fairness guarantees, with safeguards sufficient to reasonably ensure compliance with the four-fifths rule ⁴. They had also agreed in advance not to question the choice of fairness objective or metric, and did not audit the vendor's annual back-testing of deployed models. Differential validity, intersectional fairness and non-EEOC groups were outside the agreed scope. That is a narrow question answered honestly, in 2020, for one product, and it is not a finding that the tool is fair, valid or job-related.

Ask for the full report and the last retrain date

Ask for the auditor's full report, and ask for the date of the last model change. A compliant summary already has to name any category the auditor dropped under the 2% rule, with that category's own counts and the auditor's justification ¹, so check first whether the one you were sent does. The full report is where the method sits. A summary published before a retrain describes a system that no longer exists, and nobody volunteers that.

New York State auditors re-reviewed 22 employers that the city's own compliance sweep had largely cleared, using only public information, and identified at least 17 instances of potential non-compliance where the city had identified one: three audits not performed by an independent auditor, five missing proper selection-rate and impact-ratio calculations, and nine failing to properly explain their use of historical data ⁵. Those 22 employers were pre-selected by outside researchers precisely because they raised questions, so the rate does not generalise, and none of it has been adjudicated. The failure modes are the useful part, and they are the three things to check yourself: independence, the arithmetic, and whose data it ran on.

Send both requests to every vendor already in the stack, not only to the one under evaluation. Renewal is the moment nobody re-reads anything, and an audit that was current at purchase ages out on a one-year clock ². Assembling the file a regulator would ask for is a longer list: what an assessment vendor has to hand over.

Why a clean ratio can still be uninformative

Because the four-fifths ratio was never a pass mark, and because a ratio computed on small numbers swings on single decisions. The agencies that wrote the rule said in their own 1979 interpretive questions and answers that it is a rule of thumb and not intended as a legal definition, but a practical means of keeping enforcement attention on serious discrepancies ⁶. Asked whether it means the guidelines tolerate up to 20% discrimination, they answered no.

The reverse inference gets made anyway, in both directions. Clearing 80% is not a clean bill of health, and falling below it is not a finding of discrimination ⁶; both readings mistake a screening heuristic for a verdict. Sample size compounds it. The same 1979 document says it is generally inappropriate to require validity evidence or take enforcement action where the numbers are small enough that selecting one different person for one job would flip which group has the higher rate, and its worked example is three men and one woman hired from a pool of twenty men and ten women: the ratio trips four-fifths and the count is still too small to support a determination ⁶. That is how an audit can be compliant and uninformative at the same time, and why the number deserves its raw counts printed next to it wherever it is quoted.

So the honest summary for whoever signs the contract has two halves. Here is what the audit measured, on whose data, on what date. Here is what it did not measure, and here is the evidence being gathered instead. The second half is where the work sits, and it is the employer's work: the audit describes the tool, and the exposure runs on how the tool is configured and used.

On Monday, send the two requests, then write that half-page: categories covered, data source, audit date, cells the auditor could not fill, and the three questions the document does not answer. It fits on one page, it survives the renewal, and it is the thing somebody will look for in a year.

Read the evidence >