One US Jurisdiction Requires a Bias Audit. Testing Matters in the Rest Anyway.

Which employers actually owe an audit?

One city. Local Law 144 bars an employer or employment agency from using an automated employment decision tool on a New York City candidate unless a bias audit was done within the prior year, a summary of the results is posted publicly, and the candidate got notice at least 10 business days before the tool is used ¹. No state or federal law matches that.

Two states come close without getting there. Illinois amended its Human Rights Act effective January 1, 2026 to make it a civil rights violation to use AI that has the effect of discriminating, and added a notice duty, but it requires no audit ⁶. Colorado's original AI Act was repealed before it ever applied, and the disclosure-oriented law that replaced it does not start until January 1, 2027 ⁷.

The law took effect January 1, 2023, and enforcement began July 5, 2023 ¹. The notice can live on the careers page instead of going to each candidate, and posting it there starts the clock regardless of when a particular requisition opens.

Scope is the part that surprises people. The rule reaches a tool that substantially assists or replaces discretionary decision-making, and the employer decides in the first instance whether its own tool clears that bar. An employer treating a sorted list as one input to a recruiter's judgment can conclude it sits outside the rule, and no agency blesses that call in advance, so whether your tool counts as an AEDT is a determination you make and then have to defend.

Write it down when you make it. A Cornell team recorded what 391 employers publicly posted in late 2023: 18 had a bias audit report up, and 13 had a transparency notice ⁴. The authors called that null compliance rather than non-compliance, precisely because an absent audit can mean the employer concluded the law did not reach its tool. If that is your conclusion, the file should hold the reasoning, the date and the person who signed it. The question arrives later than the decision does.

What does a Local Law 144 audit actually measure?

Group selection rates, and nothing else. The DCWP rules require an independent auditor to compute a selection rate and an impact ratio for each category, separately for sex, for race and ethnicity, and for the intersectional categories ². Nothing in that arithmetic asks whether the tool predicts anything, whether it is job related, or whether any individual was treated fairly.

Two carve-outs decide what the number covers. An auditor may drop any category representing less than 2% of the audit data from the impact-ratio calculation, which most often removes the smallest groups in the pool, the ones the exercise exists to protect ². And an employer that has never used the tool may rely on an audit built entirely on other employers' historical data, or on synthetic data where too little real data exists ².

An audit is also silent about the people the tool never saw. An impact ratio computed on one tool describes that tool's outputs, not the outcome of your process, and in the same quarter those two numbers can point in opposite directions.

So a posted summary can be accurate, current, compliant, and about somebody else's applicants. That gap between the certificate and the evidence is why what to ask a vendor that says it is bias tested goes well past reading the audit they hand you. Ask for the population the audit ran on, the date range, the categories dropped under the 2% rule, and whether any of the data came from your own use of the tool.

Why run testing where no law requires it?

Because the result becomes evidence either way. California amended its FEHA employment regulations, effective October 1, 2025, so that in a claim that an automated-decision system discriminated, evidence or the lack of evidence of anti-bias testing is relevant to the claim and to any defense, including the quality, recency and scope of the effort and the response to the results ³⁸.

That sentence cuts in both directions. An employer with no testing has nothing to show. An employer that tested, found a disparity and left the tool running has produced a dated record of knowing. The regulation weighs the response to the results, which makes the testing decision and the remediation decision one decision taken twice. The same amendments lengthened the employment-records retention period from two years to four and said automated-decision-system data is included ³, so this is a records question as much as a testing one.

Enforcement is not the reason to do it. A New York State Comptroller audit covering the first two years of enforcement, July 2023 through June 2025, found that DCWP received two AEDT complaints in the entire period ⁵. Nine of the auditors' twelve test calls to the city complaint line were never connected to the agency, so read that as a broken intake route rather than as evidence about how employers behave. The exposure that has not gone quiet is private litigation and state regulators, which is where an adverse impact audit when the vendor holds the data stops being paperwork.

Decide who commissions the test before it runs

Before the first analysis, settle it in writing: who the auditor is, who they report to, what the engagement covers and how often it reruns, and what happens if the numbers come back bad. Those answers are cheap now and expensive later: a spreadsheet that already exists cannot be un-created, and whether counsel directed the work is not a question you can answer retroactively. Privilege and scope turn on your jurisdiction, so take both to counsel first.

• Independence, in the auditor's own words. The rules bar anyone involved in using, developing or distributing the tool from being called independent ². Get that statement before the engagement, not attached to the finished report.
• Scope. Which tools, which requisitions, which stages, which date range. A tool-level audit and a funnel-level analysis answer different questions, and only one of them is about your candidates.
• Cadence. Annual is the New York City minimum for a covered tool. The rule does not say when a changed tool becomes a new one, so set that trigger yourself: treat a model update, a new item bank or a change to your own cut-offs as if it were a new tool, and date the decision.
• The remediation path, written before the result. Who decides, on what timeline, and what runs in the meantime.

Then run the analysis you actually need, which is your own funnel, stage by stage. What counts as adverse impact is measured on your own numbers, and that is the measurement a plaintiff asks for.

Read the evidence >