Report the Self-ID Rate Beside Every Ratio or the Ratio Is Unreadable

What response rate is high enough?

No threshold has been published, and any page that hands you one invented it. It depends on the size of the gap and on which way the unknowns could plausibly fall, and no fixed bar sees either. The federal Uniform Guidelines on Employee Selection Procedures (1978) permit impact records on a sample basis only where the sample is appropriate in terms of the applicant population and adequate in size ². The people who chose to answer are not that sample.

The question you actually need answered is narrower than it looks. You are not trying to establish the true composition of your applicant pool. You are trying to work out whether one specific conclusion, that a gate treats groups differently, survives the uncertainty in the data. How much missing data a conclusion can absorb depends on how wide the gap is, which is why a fixed bar was never the instrument. The next section tests the conclusion directly.

Test the verdict against both bounds

Run the ratio three times: once on the answered subset, once with every unknown applicant counted into one group, and once with every unknown counted into the other. You already know what happened to those applicants. The only thing missing is which group they belong to, which is what makes the bound computable.

A gate takes in 1,000 candidates and passes 250. Demographics are known for 600 of them, and the answered subset looks spotless:

The answered subset reports perfect parity. Neither bound clears 0.8. Nothing about this gate is settled, and the finding to write down is the response rate, not the ratio. Note the direction of the second and third rows: they fail on opposite sides, because the unknowns passed at a lower rate than either known group, and whichever group absorbs them takes the hit.

Moving the block wholesale is the version you can compute in a minute and defend in a meeting. The range goes wider than that. Split the same 400 instead, the 70 who passed into group A and the 330 who did not into group B, and the ratio falls to 0.28. Treat a verdict that survives the two wholesale rows as a floor under the uncertainty, and say in the write-up which assignment produced the number you are reporting.

Nobody had to invent this. Researchers who read the bias audits published under New York City's Local Law 144 ran a cruder version of it, because a published audit does not say what became of the applicants whose demographics are missing: for each group they assumed the whole missing block belonged to it, then took the best and the worst selection outcome for that block. Of the 116 audits they collected between the law taking effect in July 2023 and November 2024, 53% report at least one impact ratio below 0.8. At the level of individual impact ratios, 70% have a possible lower bound below 0.8 under that extreme assignment, against 28% when the missing applicants are assumed to look like the known ones ¹. The authors call the extreme version demonstrative and print the gentler one beside it. Read the pair as a range the published point estimates never showed, and the same range worth demanding when a vendor holds the data behind an audit you did not run.

Why aren't the missing answers random?

Because declining is a decision, and nothing guarantees that decision is independent of the demographics being asked about. A candidate weighs what they think happens to the answer, and that weighing has no reason to be uniform across the people asked or across the points at which you ask. Treating the unknown category as random is an assumption, and it is the assumption that hands the missing rows the average behavior of the people who did answer.

You can test the second half of that in your own data this afternoon. Pull the response rate at each stage separately: at application, at the assessment, at offer. If the rate climbs through the process, the same people are answering a question they refused earlier, which puts the refusal down to where it was asked. If it is flat, that is worth knowing too, and it makes one bound tighter.

What to refuse outright is filling the gap by inference. A survey experiment on the names used in correspondence audits found the congruent perception rate for Black first names was 75.0 percent with no last name, 82.5 percent with a Black last name and 66.5 percent with a white last name, and that names common among highly educated Black mothers were markedly less likely to be read as Black than names common among less educated Black mothers ³. That study tested how survey respondents perceive a name. It did not measure what employers did, and no callback gap follows from it. What does follow is that a name is a noisy indicator of the category you want, and the noise is patterned by class and education rather than scattered at random. Imputing from names or addresses replaces a known unknown with an unknown error that runs in a direction you cannot see, and it puts the employer on record assigning a protected characteristic to somebody who declined to state one. The cheaper route is to improve the ask, which is where the demographic question has to sit relative to the decision. What an analysis may touch, and what it may be used for once it exists, is a question for counsel on your own process.

Report the rate per stage, then work on raising it

Print the response rate next to every ratio, at every stage, or the number is unreadable by anyone who knows what to look for. One rate for the whole funnel is not enough: the ask sits at one point and the population changes at every gate after it, so the share of unknowns at the offer stage can look nothing like the share at application, and each stage ratio inherits its own.

The response rate cannot be broken out by group, because the group of the non-responders is the thing you do not know. What can be published is the overall rate per stage and per requisition, the composition of the answered subset, and the bounds. Anyone who hands you a response rate by race has computed something else and mislabelled it.

Four ways to raise the rate, in rough order of effect:

1. Move the ask off the application form to its own step after submission, so it is not competing with a candidate's incentive to look employable. 2. State the purpose in one sentence naming what the data is used for and who cannot see it. Longer is worse. 3. Make declining an explicit option rather than a blank, so a considered refusal is recorded as one. 4. Ask once, later, in a second place, such as the offer stage, and keep the two rates as separate measurements.

The thing never to do is make the question mandatory. It converts a voluntary instrument into a compelled one, it produces answers people did not want to give, and it buys a denominator by damaging the numerator. When the counts themselves are too small for any ratio to carry meaning, the response rate is not the first thing in your way: start with what to measure when there are too few hires.

Monday's first query is the response rate, before anything else gets pulled. It takes a minute, and it decides whether the rest of the afternoon is measurement or arithmetic performed on a guess.

See a sample report >