Your ATS Rejects Only What Your Rules Tell It To

What does an applicant tracking system actually do?

Three jobs: store, parse, filter. It receives an application, turns the attached document into structured fields, and shows recruiters whichever slice matches a query someone typed. The filtering is real and it does remove people from view. None of it is autonomous. The query, the required answers and the thresholds are configuration, and configuration has an author, a date and an intent.

Storage is the part nobody argues about. Parsing is where quiet damage happens. The system reads a document into name, employer, title, dates and skills, and anything it cannot map lands nowhere. A field that failed to parse is not a weak candidate, it is an empty box, and an empty box loses every search that requires that field. Measuring that loss is its own exercise, and only the employer can run it.

Filtering is the part that looks like judgment. A saved search returns whoever matched the string. A knockout question returns whoever answered the way the form demanded. Both compare text to text and hand back a subset, with no weighing of a person against a bar anywhere in the operation. When that subset is the only thing anyone opens, the comparison has decided the outcome, whatever the feature is called in the menu.

Some products do ship a scoring or ordering module on top of the core system, and a few arrive with it enabled. That is a separate question from what storage, parsing and filtering do, and it is worth ten minutes in the admin panel to find out whether yours came with those features switched on.

Which three rules are actually rejecting people?

Knockout answers, saved searches, and the pile nobody opened. The first two are explicit rules with authors. The third is a capacity limit that behaves exactly like a rule. Each removes a candidate before any human judgment happens, and each leaves a trace in your system that you can count today. None of the three is the vendor's product decision.

Knockout answers. A screening question with a disqualifying answer is a hiring policy executed exactly as written. The federal record holds one worked example. In 2023, three tutoring companies paid $365,000 to settle EEOC claims that they programmed their tutor application software to automatically reject female applicants aged 55 or older and male applicants aged 60 or older, turning away more than 200 qualified US-based applicants ¹. That was a hard-coded date-of-birth cutoff a person typed in. No model learned anything, and the consent decree resolved the case with no admission of liability. Read it as the clearest picture available of what a knockout rule is.

Saved searches. The UK's Information Commissioner's Office audited providers of AI sourcing, screening and selection tools between August 2023 and May 2024 and found that features in some tools could lead to discrimination by offering search functionality that let recruiters filter out candidates with certain protected characteristics ². Those were consensual audits of vendors who volunteered, and the report gives no counts, so it supports no rate. What it does establish is that the search box is part of the selection procedure, and that the subset it returns is a decision about who gets read.

The pile nobody opened. SHRM's 2021 benchmarking data puts median time-to-fill for nonexecutive roles at 44 days, with the median organization spending 5 days on screening applicants ³. The interquartile range runs from 28 to 73 days, so read the median as a shape rather than a target. Five days of screening against whatever your last req received is a capacity figure, and capacity decides the remainder. Nobody wrote that rule. It emerged, which makes it the hardest of the three to see and the cheapest of the three to change.

Audit the three rules in one afternoon

Pick one requisition that closed in the last quarter and reconstruct what happened to every application in it. You need three counts and one read: how many people the knockout questions excluded, how many the saved searches never returned, how many nobody opened, and thirty of the excluded applications read end to end by someone who knows the job.

1. Export the disposition reasons. Where the system records an auto-disposition separately from a recruiter decision, count them per rule, per requisition, and note which template the rule came from. 2. List every saved search that ran on the req. Write down the exact strings. A search nobody can reproduce is a rule nobody can defend. 3. Count the never-opened. Total applications received, minus applications with a view event. That difference is your third gate, and it usually dwarfs the first two. 4. Read thirty of the excluded. Draw ten from each bucket. Score each one as would have interviewed, would not, or cannot tell from this document.

The read is the step people skip and the only one that answers the question. A count tells you a rule fired. Thirty applications tell you whether it fired on the people it was written for. If more than two or three of the thirty would have earned a phone call, the rule is not doing what its author intended, and you are in the same position as a team whose keyword filter stopped separating anyone.

Change the rule, not the vendor

A new system inherits every knockout question the old one carried. Every rule the audit surfaced has an owner, a purpose and a date, and the fix is to give each one all three. Retire the rules nobody can justify, rewrite the ones that were standing in for something else, and put a review date on whatever survives the afternoon.

A knockout question earns its place when it names a genuine minimum requirement of the job, when a candidate can answer it truthfully in one click, and when the same requirement appears in the posting. Work authorization and a licence the role legally cannot be performed without clear that bar. A years-of-experience threshold usually does not: it is a proxy for something the team never wrote down, and it removes career changers and fast risers in the same pass. Which knockouts still earn their place is a shorter list than most application forms carry.

Saved searches need an owner and an expiry. String matching against a resume was a weak proxy for capability before generative writing tools were common and it is a weaker one now, because the vocabulary in a document no longer tells you much about who assembled it. Rotating the strings buys a quarter at most.

The deeper question sits underneath all three rules, and the audit is what makes it answerable: whether the resume screen predicts anything at this point for the roles you hire. Answer that with your own thirty-application read, then decide how much of the funnel should still turn on a document.

See a sample report >