When Everyone Clears the Bar, the Tiebreaker Is the Test

Is a bar four hundred people clear a bar at all?

Usually a filter. A requirement that four hundred people clear is a minimum rather than a standard. Most application-stage minimums read two fields: how long somebody has done the work, and how long they studied. Raising the bar on job-related grounds is the first move to try, and it is available more often than a tie of this size makes it feel.

Neither field predicts much. A 2019 meta-analysis of 81 independent samples put prehire work experience at .06 with later job performance, .11 with training performance and .00 with turnover ⁴.

Those are corrected correlations, so a near-zero result is not an artefact of under-correction, and experience with relevant tasks, jobs or occupations did no better at about .07 ⁴. The measure is experience held at hire, which is exactly what an application shows you.

Education adds about as little, by the most generous estimate the field ever published.

In Schmidt and Hunter's 1998 table, years of education raised the multiple correlation from .51 to .52, and years of job experience raised it to .54 ⁵. Both are gains on top of a cognitive ability test, which most employers are not running.

The authors' own footnote says a positive correlation between education and ability inside a pool would push the education increment toward zero ⁵.

So four hundred people clearing the bar is not evidence that four hundred people are equally good.

The pile is evidence that the screen stopped measuring somewhere above the minimum. A line that asks for AI experience can turn into the screening question everybody passes, and four hundred people clearing it is then no surprise.

Raise it on the work, then. A cut score moved because too many people cleared it has been moved for yield, and yield is not a job-related reason. The job-related version asks for a demonstrated act: a claim the candidate checked, an output they refused, a decision they kept away from the model.

A raised bar is only available where a real distinction exists, and looking for one is the question of what to screen on when every resume already looks perfect. Where none does, the pool is genuinely tied above a defensible minimum, and the live question becomes what breaks the tie.

What does a tiebreaker chosen after the applications arrive buy?

Nothing you can defend, when the tiebreaker is a criterion picked after the applications arrived. An added criterion has to do one of two things: predict performance better than the bar already does, or make the decision cheaper without changing who gets picked. School name, employer brand and keyword density arrive with no evidence of doing either, and each can bring in a demographic pattern of its own.

Whether the step felt informal is beside the point. The Uniform Guidelines define a selection procedure as any measure, combination of measures, or procedure used as a basis for any employment decision, and name "informal or casual interviews and unscored application forms" inside that definition ¹.

Ten minutes spent rereading four resumes and picking the one from the recognisable employer is a selection procedure.

The Civil Rights Act of 1991 added 42 U.S.C. 2000e-2(k), which sets the test an added tiebreaker has to pass. A practice shown to cause disparate impact survives only if the employer demonstrates it is job related for the position in question and consistent with business necessity ².

A second route to liability sits in the same subsection. A complaining party who identifies an alternative employment practice and shows the employer refused to adopt it establishes the violation without reaching the first prong. On that route the argument is about what a less discriminatory alternative actually looks like.

Title VII covers race, color, religion, sex and national origin, so age claims run under the ADEA and disability claims under the ADA, on different standards ².

Under time pressure the tiebreakers that arrive are the ones already printed on the page, including the one-to-five self-rating the application asked for. Not one of them was validated for this job.

Why a draw has a selection rate you can state in advance

Because the arithmetic is fixed before anyone applies. Draw forty from four hundred qualified applicants and every applicant in that pool holds one chance in ten, whatever group they belong to, so the expected selection rate is identical across groups. No other tiebreaker in ordinary use can be described that way ahead of time.

The four-fifths rule occupies a single paragraph of the Uniform Guidelines, 29 CFR 1607.4(D), and divides a group's selection rate by the rate of the group with the highest rate ³.

A ratio of one rate to another, then, and never a gap in percentage points. The direction gets confused constantly.

Expected is not observed, which is where the equal selection rate gets oversold. Forty drawn from four hundred will produce group rates that wobble, and on small numbers a wobble can be read as a pattern. The regulation says as much in its own paragraph: smaller differences may still be adverse impact where they are significant in both statistical and practical terms, and larger differences may not be where the numbers are small and not statistically significant ³.

The rule describes when federal enforcement agencies will look, and what the four-fifths rule settles and what it does not is a question of its own.

A draw moves the argument back to the bar. Beyond chance wobble, any disparity in who a uniform draw advances came in with the criterion everyone was screened on. Of the steps up to the draw, the bar is the one an employer can put validation evidence behind.

Research funding and school admissions already draw lots

Some research funders and some school admission authorities already draw lots, with the mechanism written down in both. The Health Research Council of New Zealand began allocating its Explorer Grants by lottery in 2013. Each assessor gives each anonymous application one yes or no on whether it is potentially transformative and viable. Every application with two or more yes assessments enters the pool the random numbers are drawn from ⁶.

Applicants took it better than the reflex predicts, and not unanimously. In a survey of Explorer Grant applicants covering 2013 to 2019, 63 percent agreed randomisation is an acceptable way to allocate those funds and 25 percent disagreed ⁶.

Read the response rate before carrying that number anywhere. The survey drew 126 replies from 325 invitations, and support ran higher among applicants who had won funding ⁶.

Researchers competing for a NZD 150,000 grant are also not job applicants.

England wrote the same mechanism into its School Admissions Code. The Code, which came into force on 1 September 2021, lets an admission authority use random allocation when a school is oversubscribed and requires the arrangements to be set out clearly and transparently ⁷.

Two operating conditions come attached, at paragraph 1.35. The draw must be supervised by someone independent of the school, and a fresh round of random allocation must be run each time a child is offered a place from a waiting list ⁷.

Both conditions transfer to hiring without translation.

An independent supervisor is what makes a draw checkable by somebody who was not in the room, and re-drawing each time stops one draw from hardening into a ranked queue nobody validated.

Neither precedent is an employment case, and the gap is worth stating plainly. No published study compares hires made by a draw above a job-related bar against hires made by a tiebreaker, at any employer. And the Uniform Guidelines describe validation rather than allocation ¹.

The two precedents supply an operating procedure. They supply no legal cover.

Run the draw for the next round, not for the offer

Draw for who advances, and leave the deciding to the stages after it. A draw at the offer asks an unvalidated mechanism to make the highest-stakes call in the process. That is the version with the thinnest defence. A draw that decides who gets the next forty minutes of assessment is rationing attention, which is what the tie was about.

• Write the bar down and validate it. A draw pushes every defensible claim up to that point onto the bar.
• Say in the posting that a draw decides who advances above the bar. A mechanism a candidate meets first in the rejection email is the version that gets resented.
• Record the pool, the number of slots, the method and the seed, so the draw can be rerun by somebody who was not in the room.
• Redraw when a slot opens instead of keeping the first draw as a standing order.

Buying nothing is the hardest objection to answer. A draw predicts nothing about the work, it can't tell an employer which of four hundred people will do it, and it hands a rejected candidate a reason that reads as luck.

An employer with a real job-related distinction still available is better off measuring it.

The four hundred applications are still on the desk, and the honest reading of them has changed.

They cleared a bar built to keep out the unqualified, not to rank the qualified, and the size of the pile is what proves it. Cutting twelve hundred applications to a shortlist is the same rationing problem at three times the volume.

Whatever picks the forty is a selection procedure. Choose it before the applications arrive.

Read the evidence >