For an Hourly Hire, AI Skill Is Overriding the Screen and Saying So

What does AI skill look like when the app never changes?

Two acts, both of them refusals of a sort: the person on the floor decides whether to do what the screen says, and decides whether to tell anyone when they do not. Nothing else in the usual definition survives contact with a fixed app. There is no model to pick, no prompt to write, no context to supply: the interface made all three of those decisions before the shift started.

Neither act is special to hourly work. An AI skill is a decision rather than a tool you have open, and a fixed app removes every decision except what to do after the screen gives its answer.

O*NET's entry for customer service representatives, occupation 43-4051.00, records what the job already asks of a person. Asked how often the work requires decisions that affect other people, financial resources, or the organization's image and reputation, 68% of respondents answered every day ¹.

The decisions predate the software. They used to be made under a script, a policy binder or a supervisor's rule of thumb. Now a model drafts the next step, and somebody on the floor still has to accept it.

Why the wrong answer is harder to see on a fixed screen

Because a fixed app hands over an instruction, not a draft. A chat window produces text a person is expected to edit, and the editing is the job. A dispatch screen produces a next stop, a triage tool produces a disposition, a demand forecast produces an order quantity. All three arrive in the grammar of a decision that has already been made.

Human factors research has a name for what happens next. Kate Goddard, Abdul Roudsari and Jeremy Wyatt, reviewing 74 studies drawn from 13,821 retrieved papers for the Journal of the American Medical Informatics Association in 2012, define automation bias as the tendency to over-rely on automation ².

The same review names the conditions that worsen it: workload, task complexity and time constraint.

An hourly shift can carry all three at once.

A 2024 preprint put a number on the act itself. Twenty-eight trained pathology experts estimated tumor cell percentages with and without an AI decision support system, and in 7% of the AI-assisted estimates, 38 of 560, an initially correct evaluation was overturned by wrong advice from the system ³.

Pathologists are not cashiers, and 28 people in one web-based experiment is a thin base. Take only the direction from it: people who had the answer right changed it because a screen disagreed, and time pressure appeared to make those reversals more severe, though not more frequent.

The harder problem is that a wrong instruction looks exactly like a right one.

In a 2023 Harvard Business School field experiment, 758 Boston Consulting Group consultants worked realistic tasks with and without GPT-4, and one task was chosen deliberately to sit outside what the model could do.

On that one task, consultants with the model were on average 19 percentage points less likely to get it right. Correct answers ran at 84.5% in the control group, and at 60% and 70% across the two AI conditions ⁴.

The model there was a 2023 one, and the line it could not cross has moved since.

The finding worth keeping sits underneath: the consultants could not tell which side of the line the task was on.

A new hire in week two, working a queue against a handle-time target, will not see it either. Something in the room has to tell them they are allowed to look.

Ask what the screen told them to do, and what they did next

Ask about a time a system told them to do something that did not match what was in front of them. One question does most of the work here, and it does not mention AI at all. A scheduling tool, a register prompt, a routing app or a printed checklist all count. What you want is one real instance.

Then keep going, because the first answer is rarely the useful one:

• What did you do in the next five minutes?
• Who did you tell, and how did you know that was the right person?
• What happened after that, and did anything change?
• When did you go along with a system you doubted and wish afterwards you had not?

A strong answer is specific about the mismatch and names both halves of it. The screen said one thing and the situation showed another: the app routed a delivery to a dock that had been chained shut since Tuesday. Or the schedule showed a shift covered by someone who had called out that morning.

A thin answer stays in the present tense. "I always double-check" is a policy statement rather than a memory, and every candidate has one ready.

The fourth question is the one worth protecting from the time pressure of a short screen. Somebody who can describe following a wrong instruction, and say what it cost, has a calibrated sense of when a screen is worth doubting.

Somebody whose story ends at the override, with no consequence in it, may be describing a decision that never left their own head.

Score the answer on two things, both of them yes or no: did the candidate name a specific mismatch with both halves in it, and did the report reach a named person. Where the round produced no instance at all, add not observed to the scale, so silence is never recorded as a no.

Do not ask which tools they have used. The app is yours, the training is yours, and a week of shifts closes that gap for almost anyone.

Check whether the floor allows an override at all

Before the next interview, find out what happens to somebody who refuses the screen. Three things decide it: whether the app carries an override control, what using it costs in the metrics the shift is judged on, and whether the escalation reaches somebody with authority to change the app. Hiring for a behavior the process punishes is an expensive way to fill a seat.

Read the last ten escalations the floor filed. If nobody can produce them, the third answer is no.

The 2012 review also names what reduces automation bias, and none of it is a personality trait. On the implementation side sit training and an explicit emphasis on user accountability. On the design side sit the position of the advice on the screen, updated confidence levels attached to the output, and giving information rather than a recommendation ².

Of the design changes on that list, showing the information instead of the recommendation is the one to push for first. A screen showing the two numbers behind an instruction invites a comparison. A screen showing only the instruction invites compliance.

Accountability needs an address before it can be emphasized to anybody. The question of who is accountable when AI-assisted work turns out wrong gets the same answer on a floor as in an office. The answer is a name, written down before anything goes wrong.

A usage dashboard answers none of the three questions. What separates heavy AI use from good AI use on a team is what the work shows. On a floor the nearest equivalent record is the escalation log and what came back from it.

When is overriding the screen the wrong instinct?

When it is a habit with no specific mismatch behind it. One large study looked at hiring managers overriding a job test, a different act from staff refusing a screen, across 15 firms hiring low-skilled service workers. The test raised completed job tenures by just over 25%. Comparing managers at the same location, a one standard deviation higher rate of hiring against the test went with 6% to 7% shorter job durations ⁵.

Overrides there were routine, at an average exception rate of 22%. The design compares managers without randomizing them, so it reports an average and says nothing about any single call.

The lesson for an interview is clean anyway. What is worth hiring is not a person who distrusts systems, which is cheap and common, but one who can say which specific thing did not match.

"The screen is usually wrong" is the same non-answer as "the screen is always right," delivered from the other side.

The O*NET record for customer service representatives describes a job that, by 68% of respondents' account, requires decisions affecting other people, money or the organization's reputation every day.

What no occupation-level record can say is who on your floor is allowed to say no. The answer is written in the app's settings, in the metric posted on the wall, and in what happens to the first person who refuses the screen and tells somebody about it.

See how it works >