Assessment design
Why a Game-Based Assessment Has No High Score to Chase
A game-based assessment like pymetrics offers no high score to game. As a 2021 cooperative audit described it, the games are framed as recording how you play rather than what you score, and the model was trained on the client's own high-performing employees. That makes the bar a shape rather than a number, and it moves between employers. Practice packs buy familiarity with the interface, which is worth something on a timed task but cannot tell you where the bar sits.
The takeThe practice-pack industry is not a scam, and it is not what it sells itself as either. What it sells is the removal of surprise: you have seen the game screens before, so the timer does not startle you. That is worth an hour of anyone's evening. What it cannot sell is a target, because, in the design the 2021 audit described, the target sits inside a model built from employees you will never meet at a company you have not joined.
Where Olive fits
Open a role and see what the work shows
If the assessment you are sent is Olive, the AI assistant is part of the work rather than something to keep quiet about: a person writes or edits six findings, among them how you framed the problem, what you delegated and what you verified, in words rather than a score. You are granted the identical report the employer reads.
Assess your shortlistWhat Does a Game-Based Assessment Record?
In the design a 2021 audit of pymetrics described, it records features derived from how you play. The audit describes a core set of twelve browser or mobile games, which the vendor presents as drawn from peer-reviewed psychological studies 1. The auditors relay the vendor's framing: the games are not meant to be won or lost, but to surface information from how people play 1.
Each game produces a number of features per player 1.
Nothing on the screen tells you what a good performance looks like, and that is the design rather than an oversight in the instructions. A game that announced its target would measure how well you aim at the target. So the invitation arrives with no rubric, no worked example, and no way to check yourself against anything. That is uncomfortable, and it is also the only honest description of what you are being asked to do. Play, and let the play be read.
The audit was also explicit that it did not assess whether the games predict job performance 4.
The name on the invitation may not be the name in the research. Harver announced on 11 August 2022 that it had completed its acquisition of pymetrics 2. A game met under one brand may be the product audited under another, and the audit's description comes from 2020 fieldwork.
Game-based assessments look different from the buyer's chair. The employer-side article on what an early-career assessment should measure presses a hiring team on whether the instrument it bought is a repackaged aptitude test, which is the question your invitation raises for you.
Why There Is No High Score to Chase
In the audited design, the comparison group is chosen by each employer, and you cannot see who is in it. There, the model is trained on gameplay from high-performing incumbent employees at the client, typically 50 to 100 people identified through job analysis 1. The out-group is sampled from the vendor's own database 1.
The trained model then sorts applicants into three tiers, and the thresholds belong to the client 1. In the design the auditors describe, they sat typically at the 50th and 70th percentiles 1. At or above the 70th was 'Highly Recommended', below the 50th 'Not Recommended', and the band between them 'Recommended' 1. Because those lines are customisable per client, the same play can land in different tiers at two employers hiring for the same title. The tier describes a comparison, not a mark.
A high score would have to be a different score at every employer.
That is the practical problem with playing a target you invented yourself. Decide the games reward boldness, play bold, and you are guessing at a model trained on people you have never met, doing a job you have not started. Guess right and you have described someone else. Guess wrong and you have described someone else again. Either way, the description that reaches the employer is not a description of you.
Invisible thresholds are not a reason to play carelessly.
Read the instructions, use any practice round the platform offers, and then play the way you would actually play. What the games can capture is only as good as the version of you that turned up to them.
What Do Practice Packs Actually Buy?
Familiarity with the interface, which is worth something on a timed task and is not a map of the target. The thresholds belong to each client and the comparison group is that client's own chosen employees 1, so no pack can tell you what either one was where you applied. What practice removes is the first minute you would otherwise spend working out what the screen wants.
The larger question sits behind the packs rather than inside them. In a 2020 paper, Raghavan, Barocas, Kleinberg and Levy reviewed 18 algorithmic pre-employment assessment vendors, working only from what the vendors' websites disclosed at the time 3. The sites generally did not make clear whether the vendors validate their models, by what method, on what data, or per client 3. Only 7 of the 18 explicitly discussed compliance or adverse impact for the assessments they sold 3.
A silent website is not proof that no validation happened 3.
The pymetrics audit is in the public record, and it came out narrower than the word audited suggests. The 2021 audit found that pymetrics faithfully implemented its stated fairness guarantees, with safeguards sufficient to reasonably ensure compliance with the four-fifths rule 4. The auditors had agreed in advance not to question the choice of fairness objective or metric, and they did not audit the annual back-testing of deployed models 4. It was one vendor, one product, in the summer of 2020 4.
Passing that audit is not a finding that the games are valid, fair, or job related 4.
A bias audit an employer publishes about its own hiring is a different document again, and how to read a published bias audit before you apply covers what that kind certifies, which is a more useful question than whether the games can be beaten.
Ask for an Alternative If You Cannot Play the Game
Ask the employer for an alternative format, in writing, and ask early. The 2022 guidance from the United States Equal Employment Opportunity Commission on the ADA and algorithmic assessment uses this exact case as its worked example 5. A business requires a 90 percent score on a gamified memory assessment, and a blind applicant who cannot play the games is rejected, though that applicant might still have a very good memory 5.
The cut score is the mechanism the guidance names. A tool screens someone out when a disability prevents them from meeting a selection criterion, or lowers their performance on it, and they lose the opportunity as a result 5. Screening out is not automatically unlawful, and an employer can still defend a criterion as job related and consistent with business necessity.
That guidance never had the force of law, was removed from eeoc.gov in January 2025, and is read here through an Internet Archive capture 5. It speaks to United States employers, so outside the United States check the law that applies to you.
Naming the barrier and the alternative is enough to make the request. The game format is a barrier because of a disability, and you are asking whether a non-game version of this stage exists. Under the same guidance, telling an employer that a medical condition may make a test difficult already counts as a request for accommodation, with no need to mention the ADA, though what counts as a request is not the same as what must be granted 5.
The guidance also lets an employer ask for supporting medical documentation where the disability and the need are not obvious 5. The employer-side article on whether ADA accommodations apply to an AI-based assessment sets out what a hiring team is told to have ready.
Send it in writing, and keep a copy.
An employer that cannot say what its games measure is telling you something about the process you would be joining. Asking costs one message before the deadline. The answer, or the absence of one, is information you keep either way.
Common questions
Can you get a higher result on a game-based assessment by practising?
Practice buys familiarity with the interface, which is worth something on a timed task, and there is no published target to practise toward. In the design covered by the 2021 audit of pymetrics, applicants were sorted against a model trained on the hiring company's own high-performing employees, with the thresholds set by that company. Which employees were used is not visible to the candidate, so nothing sold as preparation can tell you what the target is.
What do these games actually measure?
In pymetrics, as audited, features derived from how you play. The 2021 audit of pymetrics describes twelve browser or mobile games the vendor presents as drawn from peer-reviewed psychological studies, each producing a number of features per player, and relays the vendor's framing that they are not meant to be won or lost. The same audit is explicit that it did not assess whether the games predict job performance, so what those features are good for is a separate and unanswered question.
Does it matter which vendor the games come from?
It matters for what you can look up. The 2021 cooperative audit, co-written by outside researchers and pymetrics staff, covers pymetrics, and Harver announced in August 2022 that it had acquired pymetrics, so games met under the Harver name may be the audited system. Other companies sell game-based assessments too, and whether any of them has published something comparable is a question to ask rather than assume. Asking which vendor runs the stage, and what the result is used for, is reasonable before you start.
Does a published bias audit mean the assessment is fair?
No. The 2021 audit of pymetrics found the vendor faithfully implemented its stated fairness guarantees, with safeguards sufficient to reasonably ensure compliance with the four-fifths rule. The auditors had agreed in advance not to question the choice of fairness objective or metric, and the audit did not cover the annual back-testing of deployed models. Passing it is not a finding that the tool is valid, job related, or that it predicts how anyone will perform.
What happens if a disability makes the games impossible to play?
Ask the employer for an alternative format, in writing, and ask before the deadline. The 2022 EEOC guidance for United States employers uses that case as its worked example: a business requiring 90 percent on a gamified memory assessment rejects a blind applicant who cannot play the games, though that applicant might still have a very good memory. Screening out is not automatically unlawful, and an employer can still argue a criterion is job related and consistent with business necessity.
References
- 1. Building and Auditing Fair Algorithms: A Case Study in Candidate Screening ccs.neu.edu The twelve-game design and its per-player features, the model trained on 50 to 100 high-performing incumbents at the client, and the three tiers with thresholds typically at the 50th and 70th percentiles, which is this article's account of what a candidate's play is compared against.
- 2. Harver Acquires pymetrics, Further Enhancing Talent Decision Capabilities Across the Employee Lifecycle harver.com Supports the provenance point that a game-based assessment met under the Harver name today may be the pymetrics product the audit describes.
- 3. Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices arxiv.org Supports the claim that across 18 assessment vendors' websites, validation was generally not made clear and only 7 discussed compliance or adverse impact for what they sold.
- 4. Building and Auditing Fair Algorithms: A Case Study in Candidate Screening ccs.neu.edu Supports what the audit did and did not certify: faithful implementation of the stated fairness guarantees and the four-fifths rule, with the fairness objective, the metric and the back-testing of deployed models outside its agreed scope, and no assessment of whether the games predict performance.
- 5. The Americans with Disabilities Act and the Use of Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees web.archive.org The gamified memory assessment with a 90 percent cut score as the agency's worked example of screening out, the definition of screen out this article states, and the same guidance's point that saying a medical condition may make a test difficult is already a request for accommodation, with the employer free to ask for supporting documentation where the need is not obvious.
5 sources, numbered by first appearance. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.