Assessment design
Keep the Algorithm Practice, Cut the Volume
Algorithm practice still buys something, but less than it did, and not the thing volume used to buy. Models clear routine problems on public sites at high rates, and the solve rate falls as difficulty rises. Assessment vendors and employers responded by changing the round rather than dropping it: leaked-question alerts, per-test AI rules, and live formats where a person asks why you did it that way. Practise fewer problems, and practise explaining them.
The takeBoth stock answers are about volume. Grind more, or stop entirely. Neither notices that the round changed shape, which is the only thing that should change your plan. A hundred problems worked slowly, out loud, with the cost named and one constraint moved, prepares you better for a live conversation than a thousand solved in silence. The count was never the skill. It was a proxy for recall, and recall is the part that got cheap.
Where Olive fits
Open a role and see what the work shows
If the assessment an employer sends is Olive, the AI assistant is part of the task rather than something to work around: a person writes or edits the findings on what happened, in words, covering how the problem was framed, what got delegated, and what was verified. The candidate is granted the identical report, free.
Assess your shortlistDoes the practice still buy anything?
Yes, and less than it used to. Two studies bound the change. Evaluated on 115 Python problem statements taken from HackerRank, OpenAI's Codex solved 96% of them with no worked examples in the prompt, and the authors report clear signs the model was reproducing memorised code rather than writing it 1. That was 2022, on a public problem bank.
The gradient matters more than the headline. In a 2024 study that submitted model-written solutions to LeetCode, GPT-3.5-turbo solved 92% of easy problems, 79% of medium and 51% of hard 2. Public problems, a superseded model, and correctness measured by whether the submission passed the tests. Not code quality, not design judgment, not whether anyone could explain it afterwards.
Both numbers are floors on what current models can do, and neither study touched work in a real repository 12.
So the part of practice that was pure recall lost most of its market value. Recognising a problem type in ten seconds is still pleasant. It no longer separates you from the next applicant, because the next applicant can ask for the pattern and get it back in a sentence.
What survived is everything the solve never measured. The cost of the approach in time and space, said out loud, and the trade-off you took against the one you rejected. Then the moment someone changes the input size, when a memorised answer stops holding and you have to think in front of a person. Those were always the interesting parts of the exercise. They were hard to grade at scale, so silent, auto-graded screens left them out. A live round can grade them, one candidate at a time.
What did employers do instead of dropping the format?
They changed the round, largely through options their assessment vendors added. HackerRank's Screen page lists integrity signals including tab proctoring, copy and paste tracking, multiple-monitor detection, leaked question alerts and full session replay 3. The same page states the rule per test: if a test is designed to allow AI, everyone may use it, and if it says not to, using it gets flagged as suspicious activity 3.
Everything above comes from HackerRank's description of its own product, with no accuracy or false-positive rate published. A flag goes to the employer and is not a verdict, and which signals you meet depends on what the employer switched on 3.
The larger change is a new format beside the silent solve. In December 2025 Karat launched NextGen Interviews, in which engineers work on complex multi-file projects with an integrated AI assistant while a live Karat Interview Engineer probes reasoning, trade-offs and judgment in real time 4. That is a launch announcement about one product line, and whether you meet it depends on the employer. The claim that this reveals genuine engineering ability is the vendor's own, with no validity evidence on the page 4.
HackerRank's March 2025 recap describes an AI Interviewer for first-round interviews that evaluates depth of knowledge, problem-solving approach, code quality and ability to work with AI, offering hints when a candidate is stuck 6. The post says it had conducted over 12,000 interviews, a vendor count that does not say whether those were practice sessions with developers or live screens for employers 6.
Some of the change runs the other way. Asked in June 2025 how Google's coding interviews had changed, Sundar Pichai said Google is making sure it will introduce at least one round of in-person interviews, to check the fundamentals are there 5. He added that if you can use these tools to generate better code, that is an asset 5.
Intent stated on a podcast is not a published policy.
Employers are hearing much the same thing from their side of the table. The employer-facing piece on what replaces an online assessment a model solves in seconds argues for a task where the model's answer is only the start, and live coding, take-home or AI-allowed work sample weighs the formats against each other.
Cut the volume and change the drill
Solve fewer problems and spend the time you save on the part that now gets watched. For each problem, say the approach out loud before you write anything, name the time and space cost when you finish, then change one constraint and adapt. Three problems worked that way teach more than thirty solved in silence.
The last step is the one most people skip:
- The input no longer fits in memory, so the version that sorts everything up front stops working.
- The data arrives as a stream and you cannot see it twice.
- Duplicates are allowed, and the set you were relying on quietly loses them.
- The interviewer wants the second-best answer and the reason you did not take it.
That is the closest thing at your desk to what a live interviewer does. A coaching study that tested levels points the same way, loosely. Tross and Maurer assigned 144 participants to three levels of coaching before a structured past-behaviour interview (an introduction, then awareness training, then skill training on top), and the more comprehensive the coaching, the higher the ratings 7. That was a research interview with human raters, no effect sizes in the abstract and no code in it, so read it as a pointer about practice, not about LeetCode.
Reading a solution is not practice; saying one out loud, badly at first, is.
Keep a small set and revisit it. Twenty problems you can rebuild from memory, explain in two minutes each, and adapt when a constraint moves will carry you further than a list of four hundred you have seen once. What a live round adds is the moment after the interviewer changes the problem.
None of this asks for more hours, only a slower loop over a smaller set.
What if the round lets you use AI?
Read the instructions for that specific test, because the same platform allows AI on some tests and flags it on others 3. When it is allowed, the assistant is not a shortcut past the interview. It is part of what is being looked at: the Karat format puts a live engineer beside the assistant, asking why you took that path 4.
That changes what preparation means. You are being read on how you frame the problem before generating anything, and on what you check before you trust what comes back. Typing speed was never the point, and it is less of one now.
The employer-side guide to running a coding interview with AI open is worth reading for that reason: it is what the person across from you was told to look for. The candidate-side version, when AI is allowed in the coding round, covers what the interviewer is watching for once the assistant is open.
Ask which kind it is before you start, in one line.
If the instructions say no AI, follow them. The take-home that says no AI covers what tends to happen when people don't, and what to ask when the rule does not fit the task.
The hours are not wasted.
They buy the ability to hold an answer up and turn it around while somebody watches, which is a different thing than the advice was written for.
Common questions
Should I stop practising algorithm problems entirely?
No. Vendors and employers changed the round rather than retiring it, and platforms still run coding screens with per-test rules about AI. What has lost value is volume for its own sake. Recall of a pattern is now available to anyone who asks for it, so the hours are better spent on a smaller set worked slowly: the approach said out loud, the time and space cost named, and one constraint changed so you have to adapt.
How many problems is enough?
Fewer than the advice usually says, worked in more depth. A set you can rebuild from memory and explain in two minutes each does more for a live round than a long list seen once. The useful test is not whether you recognise a problem but what you do in the moments after someone changes it, since that is the part a live interviewer is there to watch.
Do coding assessments still get used at all?
Yes, and vendors have kept building on them. HackerRank's Screen page lists leaked question alerts, tab proctoring, copy and paste tracking, multiple-monitor detection and full session replay, and it sets the AI rule per test 3. Karat launched a format in December 2025 where engineers work on multi-file projects with an integrated AI assistant while a live Interview Engineer probes reasoning and trade-offs 4. Both are vendor descriptions of their own products, and which one you meet depends on the employer.
Am I allowed to use AI during a coding assessment?
It depends on that test, and the instructions are where the answer lives. HackerRank's Screen page frames the rule per test: if a test is designed to allow AI, everyone may use it, and if it is explicitly instructed not to, using it gets flagged as suspicious activity 3. If nothing on the screen says either way, ask before you start. A one-line question to the recruiter is a reasonable thing to send.
Are in-person interview rounds coming back?
At some employers, possibly. Asked in June 2025 how Google's coding interviews had changed, Sundar Pichai said Google is making sure it will introduce at least one round of in-person interviews, to check the fundamentals are there, and added that using these tools to generate better code is an asset 5. That is a chief executive's statement of intent on a podcast rather than a published policy, and it does not say which roles or when.
References
- 1. Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation arxiv.org Supports the claim that a model clears most problems taken from a public problem bank, and the memorisation finding that makes the number a statement about lookup rather than capability.
- 2. Evaluating ChatGPT-3.5 Efficiency in Solving Coding Problems of Different Complexity Levels: An Empirical Analysis arxiv.org Supports the easy, medium and hard solve rates on LeetCode and the point that the rate falls as algorithmic difficulty rises.
- 3. HackerRank Screen (product page) hackerrank.com Supports the list of integrity signals the platform says it uses, and the per-test AI rule the article tells the reader to look for in the instructions.
- 4. Karat launches NextGen Interviews: the first human-led, AI-enabled talent evaluation solution karat.com Supports the description of an AI-allowed coding interview in which a live Interview Engineer probes reasoning and trade-offs while the candidate works with an assistant.
- 5. Transcript for Sundar Pichai: CEO of Google and Alphabet | Lex Fridman Podcast #471 lexfridman.com Supports the in-person round claim in the words of the chief executive who said it, and the hedge that this is stated intent rather than published policy.
- 6. HackerRank's AI Day 2025: Product Launch Recap hackerrank.com Supports the claim that a first-round interview product describes ability to work with AI as one of the things it evaluates, with the vendor's own interview count.
- 7. The effect of coaching interviewees on subsequent interview performance in structured experience-based interviews doi.org Supports the claim that rehearsing the act of answering, rather than only reading about the format, is what raised ratings in the one dose-response coaching study.
7 sources, numbered by first appearance. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.