Should AI Be Its Own Competency in the Framework?

What is the split test, and how do you run it?

One question, asked of one competency at a time: did AI change the method, or did it change the deliverable? If the output a person is accountable for is the same document, decision or build, and only the route to it moved, the competency you already have is the right home. Rewrite its anchors. If what gets produced or approved is now a different thing, no existing definition covers it, and it needs its own row.

Run it with the job description in front of you rather than the framework:

1. Name the deliverable the person is accountable for today. Not the task list: the object that leaves their hands and that somebody else acts on. 2. Name what that object was two years ago. Same object by a different route, or a different object? 3. Same object: stay inside the existing competency. Rewrite its behavioral anchors so they describe how the work is done now, and leave the competency name alone. 4. Different object: write a new competency. None of the definitions you have was written about it, and stretching one to fit produces an anchor nobody can rate.

Both blanket answers fail, in opposite directions. Bolting one company-wide AI competency onto every scorecard produces a row that reads "uses AI tools effectively", which is unratable by a manager, undisputable by an employee, and quietly ignored by the second review cycle. Folding it in everywhere hides the two or three functions where the job genuinely changed, because the old anchors still look satisfiable and nobody is forced to rewrite them.

The test needs one input the framework does not contain: which roles the change actually reached. Which roles need AI skills is the list to settle first, because a framework decision taken before it exists is a guess applied uniformly to people the guess does not fit.

Rewrite one competency both ways

Take Analysis and Problem Solving, which nearly every framework already carries. Below it is written twice on the same job (a financial analyst producing an investment memo), first folded in, then standalone. The competency name is identical in both. What differs is where the AI behavior is described, and therefore what happens to the rest of the framework.

Folded in (Analysis and Problem Solving, anchors revised):

Standalone (Analysis and Problem Solving untouched, plus a new row):

For this analyst, the folded-in version wins. The memo is still the memo, the reader still acts on it the same way, and the anchors now describe how it gets made. The standalone version creates a second place to write "checked the number," and a manager rating both rows will double-count one act or split it arbitrarily.

What moved for that analyst is the method, and the direction of the move is measurable. A survey of 319 knowledge workers describing 936 real tasks found effort shifting from gathering information toward verifying it, from problem-solving toward integrating a partial response, and from execution toward stewarding a task the model is doing part of ¹. Verification, integration and stewardship are not new competencies. They are the old ones with the weight redistributed, which is exactly what an anchor rewrite is for.

Now change the job. A support lead who no longer writes replies but approves drafted ones has a different object leaving their hands: not a reply, but a decision about somebody else's reasoning, taken sixty times an hour. Analysis and Problem Solving was never written about that, and no anchor rewrite reaches it. That role gets the standalone row, and the standalone row is the one that survives review, because you can point at the act it names. What good AI use looks like in practice is the vocabulary both versions are drawing on.

Why does the answer differ function by function?

Because AI reaches different fractions of different jobs, and the fraction decides the verdict. Anthropic's index of roughly a million assistant conversations mapped to occupational tasks found about 36% of jobs showing AI use across at least a quarter of their tasks, but only around 4% across at least three-quarters ². A framework decision taken once, at company level, is applied to both ends of that distribution.

The same index splits usage between augmentation, where the person stays in the loop, and automation, where the task is handed over: roughly 57% against 43% ². That split is the split test in aggregate. Augmentation is a method change and belongs inside existing competencies; automation moves the deliverable and is what a new competency is for. Read the figures as what people brought to one assistant rather than as what your firm does. The mapping measures conversations, not staffed work.

Even inside one function the change is uneven. In a study of 5,179 customer support agents, access to a conversational assistant raised issues resolved per hour by 14% on average, 34% among novice and low-skilled agents, and close to nothing among the most experienced ³. A single framework row rated on a single scale is being asked to describe a capability that moved four times as much for one cohort as for another.

So run the test per function and expect a mixed answer:

A framework that forces one answer across that table will be wrong in half the organization, and wrong in the direction nobody notices: the folded-in roles look fine either way, and the changed-deliverable roles get a row that describes a job they stopped doing. What the AI actually does in the role is the evidence this table has to be rebuilt from, function by function, with your own job descriptions.

Where does an AI competency sit in the framework?

In a tiered model, position settles most of the argument. The Department of Labor's Building Blocks Model runs six tiers: personal effectiveness, academic, workplace, industry-wide technical, industry-sector technical, and occupation-specific or managerial ⁴. Method changes belong at tier three, where workplace competencies apply to everyone. Deliverable changes belong at tiers five and six, where competencies are allowed to differ by occupation and are expected to.

That placement resolves the objection most often raised against a standalone row, which is that it will be generic. A tier-three competency is generic by design, and "uses an assistant within the disclosure rules and states when output was model-produced" is a perfectly good tier-three anchor. The failure is writing an occupation-specific behavior at tier three, where it applies to people it does not fit, or writing a generic one at tier six and calling it rigorous.

Three tests before a proposed AI competency goes into the framework:

• Can you write an anchor a manager could observe on an ordinary Tuesday? "Demanded a source for the revenue claim and opened it" is observable. "Uses AI tools effectively" is a value statement wearing a competency's clothes.
• Can anyone fail it? A row nobody can be rated below on is decoration, and it costs a real slot in a review conversation that is already too short.
• Would deleting it change any rating? If the same person receives the same rating with and without the row, it duplicates something already in the framework, and the honest fix is the anchor rewrite rather than the new row.

The wording carries into hiring whether you intend it to or not, because the competency framework is where job requirements are written from. Writing AI skills into job requirements is the downstream surface, and a vague internal anchor becomes a vague external requirement roughly one quarter later.

What breaks when the framework decides pay or promotion?

The competency stops being a description and becomes a selection procedure. The moment a rating moves someone (a promotion, a band, a reassignment onto an AI-heavy team), it carries the same expectations as a hiring test: job-related, applied consistently, and reviewed if the pattern of who passes turns out uneven ⁵. That is an argument for observable anchors, not an argument against the row.

Three things follow, and none of them is expensive at design time.

• A generic anchor is the exposure. "Uses AI effectively" cannot be shown to be job-related, because nobody can state what it required. The specific anchor is both the fairer instrument and the defensible one.
• A new competency has no rating history. The first cycle produces numbers with nothing to compare them against. Announce that cycle as a diagnostic before it runs, and keep it out of pay until there is a distribution to read.
• Self-assessment is the weakest possible input, and it is the default one for a brand-new row. Sixteen experienced developers working real issues in their own repositories took 19% longer with AI tools allowed and believed afterwards they had been 20% faster ⁶. People misread their own AI-assisted throughput by that margin in work they know intimately.

Then the limit worth stating plainly, because it is the one a framework cannot fix. A competency framework says what good looks like. It does not observe anybody doing it. The rating still comes from a manager reconstructing work they mostly did not watch, on a behavior (what was refused, what was checked, what was kept) that is visible only while it happens and invisible in the finished deliverable. That gap is why a defensible AI proficiency bar has to name evidence rather than adjectives, and why assessment scores and actual performance part company when the instrument grades output instead of acts.

So: run the split test per competency, write anchors somebody could be marked down on, place them by tier, and decide separately, and later, whether any of it touches money.

See the benchmarks >