What Do You Need From an Assessment Vendor for a Bias Audit or EEOC Inquiry?

Who Owes the Documentation, You or the Vendor?

You do. The Uniform Guidelines on Employee Selection Procedures put the obligation on the user of a selection procedure (the employer making the decision) to maintain records of adverse impact and, where impact appears, evidence of validity ¹. A procedure that has adverse impact is treated as discriminatory unless it has been validated ³. The vendor holds the evidence. The duty to produce it is yours.

That one fact reorders a procurement. You are not checking whether the vendor is compliant, because compliance is not a property a vendor can hold on your behalf. You are buying a file you may one day hand to an investigator, to opposing counsel, or to your own general counsel, and the only version of that file you will ever have is the one your contract obliged the vendor to give you.

It also explains why so many assessment sales calls end in a badge rather than a document. A badge is cheap and describes the vendor. A validity report describes a job, ideally yours, and costs an industrial-organizational psychologist weeks. Most of that gap hides in the difference between a bias audit and a validation study: an audit measures who came through at what rate and says nothing about whether the thing being measured matters for the work.

Ask for These Documents by Name

Send the list as a procurement request rather than a demo question, because each item has a named owner at the vendor and a delivery time measured in weeks. Seven of the nine items in a content-validity report are marked essential, and a report missing one is treated as incomplete unless you can affirmatively demonstrate why it was unavailable or irrelevant ¹. Ask in writing, and date the request.

1. The validity report, for the jobs you assess. Who writes it: the vendor's industrial-organizational psychologist, or the outside firm retained for the study. Red flag: "the platform is validated," with no report and no occupation named. Validity is evidence about a procedure used for a job, not a property of software. 2. The job analysis underneath it. Who writes it: the same psychologist, usually working with subject-matter experts who do the job. Red flag: an off-the-shelf competency framework. The report has to completely describe the work behaviors, their associated tasks, the work products they produce, and how criticality was measured ¹. 3. The map from each assessment task to a work behavior. Who writes it: the vendor's assessment designer. Red flag: a rubric with dimension names and no line back to the job. The user must be able to identify the work behavior each item is intended to sample, and to compare the manner, setting and complexity of the exercise against the real work situation ¹. 4. The derivation of the cutoff you will run. Who writes it: the study author, with whoever configures your account applying it. Red flag: a benchmark score with no derivation. Where a cutoff is used, the report must describe how normal expectations of proficiency within the workforce were determined and how the cutoff was set; where scores are used for ranking, it must show that a higher score is likely to mean better job performance ¹. 5. The alternatives the vendor investigated. Who writes it: the study author. Red flag: none were investigated. Looking for suitable alternative procedures, and alternative ways of using the procedure, with as little adverse impact as possible is part of the validity study rather than a separate courtesy ³, and the report must describe the scope, method, findings and conclusions ¹. 6. Adverse impact figures broken out by job and by threshold. Who writes it: the vendor's data team for their pool, and your own team for your funnel. Red flag: a single model-level ratio. Records have to disclose the impact of a selection procedure by sex and by the listed race and ethnic groups, and the four-fifths rule cuts both ways: smaller differences can still be adverse impact where they are significant in statistical and practical terms, and larger ones may not be where the numbers are small ⁴. 7. The auditor's engagement letter and the audit date. Who writes it: the auditor, not the vendor. Red flag: a logo on a slide. Where New York City's Local Law 144 applies, an independent auditor cannot have been involved in using, developing or distributing the tool, cannot be in an employment relationship with the vendor or the employer, and cannot hold a direct or material indirect financial interest in either ⁶. 8. The accommodation and alternative-process procedure. Who writes it: the vendor's accessibility or compliance lead. Red flag: an accessibility statement about screen readers. What you need is the written path a candidate takes to request a change, which changes are available, and who decides. That is the ground covered in accommodations on an AI-open assessment.

One item costs nothing and tells you the most. The Guidelines mark as essential the name, mailing address and telephone number of the person who can be contacted for further information about the validity study ¹. A vendor that answers with a shared support address does not have a study with an author.

Why One Validity Study Rarely Covers Six Job Families

Because validity evidence travels only as far as job similarity carries it. To rely on a study run somewhere else, incumbents in your job and in the studied job must perform substantially the same major work behaviors, shown by job analyses on both jobs, and the studies must include fairness evidence for each group that is a significant factor in your relevant labor market ². Software developers do not stand in for underwriters by assertion.

Run that against a rollout. If one assessment is going in front of six job families and the vendor holds one study on one of them, that study is most of your defense for that family and close to none for the other five. It is not a gap you can paper over in the contract either, because the missing artifact is a job analysis of your job, and that exists only if someone performs it.

The Guidelines also price the borrowing in paperwork. Evidence taken from another user's study has to arrive with a description of the important work behaviors of your job and the basis for calling them important, a full account of why those behaviors are the same as in the original study, a comparison of applicant-pool characteristics (including race, sex and ethnic composition) against your labor market, and a bibliography of validity reports for the job in question. All of it is marked essential ¹. And where variables in the other study are likely to affect validity significantly, you cannot rely on it at all ².

So the useful procurement question is not whether the tool is validated. It is which occupations the study covered, and what adding yours would cost and take. Stage the rollout family by family, starting with the one the evidence already reaches. The twelve questions that separate a bias-tested claim from a report belong in the same section of the RFP.

What Do You Have to Keep, and for How Long?

One year from the making of the record or the personnel action, whichever is later. And once a charge has been filed, every personnel record relevant to it until final disposition, which the rule says expressly includes test papers completed by the unsuccessful applicant and by every other candidate for the same position ⁵. A vendor's default deletion window is usually shorter than that.

Which makes retention a contract term rather than a setting. Ask for a legal hold you can trigger yourself, an export of the underlying candidate records in a form that opens without the vendor, and a written commitment that a subscription ending does not end your access to what you already collected. A dashboard view is not a record you can produce.

Your own impact records are separate and continuous. You should maintain and have available for inspection records disclosing the impact your selection procedures have by sex and by the listed race and ethnic groups ⁴, with adverse impact determinations made at least annually for each group constituting at least two percent of the labor force in the relevant labor area or of the applicable workforce ¹. That is your funnel, not the vendor's pool, and nobody else can build it for you.

The same split runs through the city rules. Where Local Law 144 applies, the employer publishes the summary of results on its own careers page before use and gives candidates notice at least ten business days ahead, with instructions for requesting an alternative process or an accommodation ⁶. A vendor offering to handle compliance is offering something it cannot deliver, because those duties attach to your site and your process.

What to Do When the Vendor Can't Produce It

Write the refusal down and price the tool as undocumented for your roles. Vendors decline for two different reasons. Either the artifact does not exist in the form the deck implied, or it exists and does not say what the deck said. Both are decision-grade. Ask for the answer in writing, file it with the procurement record, and take it to counsel before anything is signed.

There is a contractible middle path, and good vendors take it. Name the documents in the agreement as dated deliverables. Add a legal-hold and export clause, a cooperation clause binding the vendor to support an audit or a charge response at a stated rate, and a right to terminate if a named document is not delivered by its date. Vendors who have the file will sign that. Vendors who do not will negotiate the clause instead of the price, which is its own answer.

Then pilot on one job family, run beside your existing round rather than in place of it, so you are comparing outputs rather than making decisions on evidence you cannot yet document. That also produces the artifact you will want later: a dated study on a named occupation with your applicants in it, rather than a certificate. Bring the same list when you sit down with counsel, because what your legal team will actually ask for is this file, not a summary of it.

See what gets scored >