Screening
A Prompt Library Is Cheap to Copy. Ask What It Replaced.
A prompt library and a set of custom GPTs are copyable text, so the artifact alone carries almost no evidence about the candidate who sent it. What carries evidence sits around it: which recurring task each prompt replaced, who else ran it, what broke when the model changed, and how the author found out it was producing bad output. Ask for the version that stopped working. Twenty minutes of that does more to separate somebody who built a library from somebody who collected one than the files can.
The takeA prompt library looks like craft and costs almost nothing to assemble. Anybody can fork a published prompt collection in an afternoon and arrive with a portfolio, which does not make prompt work fake. The ease of assembly moves the burden of proof. The person who ran something real can often name a failure once asked, and the person who collected will keep describing quality. Treat the folder as an agenda for a conversation rather than a credential. Declining the conversation is the only answer that should end anything.
Where Olive fits
Open a role and see what the work shows
No portfolio can tell you who wrote it, so Olive assesses the person instead of the artifact: a 40-to-60-minute occupational assignment worked alongside an AI assistant, returned as six findings with a timestamped excerpt behind each one. The candidate is granted the same report the employer reads.
Assess your shortlistWhat does a prompt library actually prove on its own?
A prompt library proves almost nothing on its own about the person who sent it. A prompt is a text file, and a custom GPT can be as little as a name, an instruction block and a few uploaded files. Both are quick to reproduce from somebody else's original. Prompt collections are published and free, so a weekend is enough to assemble a folder that reads like two years of practice.
The custom GPTs are the part that sounds most like engineering. Configuring one can mean no more than writing instructions, attaching files and giving the result a name. Nothing was trained.
What one instruction block did to one model's rankings has been measured, on the same kind of artifact a candidate hands over. Researchers at the University of Washington built a custom GPT instructed on diversity and disability-justice principles and pointed it at resumes: it ranked an enhanced CV first in 37 of 70 trials, against plain GPT-4's 15 1.
The same custom GPT still put the strictly stronger CV second about half the time, on a design of one CV, one job description and ten trials per condition 1.
Resume screening is not portfolio review, so take the narrow part. An instruction moves a result. It does not stabilize one.
But copyability is not an accusation, and reading it as one gets the screen wrong. A candidate who sends a prompt library may have written every line of it, and the files cannot tell you whether they did. Telling who actually did the work on an AI-built portfolio runs into the same wall, and it clears the same way: stop reading the artifact and start asking about the work behind it.
Ask what the prompt displaced, and who else ran it
Start with displacement, because a prompt that replaced nothing is a demo. Take the three the candidate leads with and ask, for each one, what the task was before, who did it, how often it ran and how long it took. A real answer arrives with a calendar attached: the Monday report, the intake queue, the reconciliation nobody wanted.
Usage is the second fact and the harder one to invent. Ask how many people other than the author ran the prompt, and what happened the first time somebody else did.
A prompt written by one person and used by one person is a personal shortcut.
Nothing is wrong with a personal shortcut. It simply carries less than it appears to: evidence of a habit, not of work other people depended on.
Then ask what it cost to keep. A library that ran for a year has maintenance in it: prompts that were edited, prompts that were deleted, and a note somewhere saying which model version each one assumes.
Ask which prompt in the folder is oldest, and whether it still works.
The oldest prompt is diagnostic on its own. Somebody who ran a library has an answer, because the day it stopped working cost them an afternoon. Somebody who collected the folder in March has never had the question come up. Asking what broke before crediting a built-an-agent line is the same move against a different claim.
Ask for the version that stopped working
One question does more work than the rest of the screen: which prompt used to work, stopped, and what happened next. Model updates can break prompts on a schedule the author does not control, so somebody who ran a library for six months may well have a broken prompt to describe. A real answer names a date, a symptom and a fix.
Listen hardest to how they found out. A strong answer has a detection story in it: a colleague flagged an output, a check failed, a number came back wrong and somebody traced it backward.
A weak answer is that the writing started feeling worse.
Self-estimates of how much AI speeds up work have been tested against the clock. In a randomized trial run by METR, 16 experienced open-source developers completed 246 real tasks on repositories they had worked on for about five years. With early-2025 AI tools allowed, they finished 19% slower 2.
The developers had forecast a 24% speedup before starting. After finishing, they still estimated a 20% speedup 2.
The METR sample is small, 16 developers in one setting, and none of it measures prompt libraries. What travels is the gap between belief and measurement: somebody telling you a prompt saves four hours a week is reporting a belief, and in that one trial, belief ran the wrong way.
So ask for the working instead of the result. In a 2026 study, engineering students were allowed to use ChatGPT during take-home open-book exams and were required to submit their interaction transcripts alongside their solutions 3.
The researcher reported that some of the strongest evidence of reasoning appeared where students evaluated incorrect or incomplete AI responses. A correct final answer, the study concluded, may no longer be sufficient evidence of comprehension 3.
Scope is one course, with no control group, no comparison of grades, and a course grade at stake rather than a job. The study points a direction rather than validating a method.
Which claims in a prompt portfolio can you check?
Two of them, in most packets. Whether a custom GPT opens and behaves the way the candidate described takes five minutes, and whether a named colleague remembers a prompt existing takes one reference call. The rest of a prompt portfolio is unfalsifiable by construction.
Making a claim checkable may move how often it gets inflated. One orthopaedic surgery residency program verified every publication its applicants listed and found 13 misrepresented citations out of 1,100 in the 2016-2017 cycle, or 1.18% 4.
The same program's earlier studies had reported 18.0% in 1999 and 20.6% in 2007, on denominators that are not identical, and its authors say that adding the PubMed identifier to the application form has possibly encouraged veracity 4.
Authorship of a paper is uniquely easy to verify, and the identifier explanation is the authors' hypothesis rather than a tested cause. The transferable part is narrower and still worth having: ask for the field somebody can look up.
What to request, in the order it is cheap to check:
- A working link to each custom GPT, so its behavior can be tried rather than described.
- The name of one colleague who used a prompt, and permission to ask them what it was for.
- The prompt as it stood before the last edit, plus the reason for the edit.
- The repository, document or ticket where the library actually lived.
Reading the packet stops being a neutral act the moment it moves a decision.
"Any measure, combination of measures, or procedure used as a basis for any employment decision" counts as a selection procedure under the Uniform Guidelines on Employee Selection Procedures, the U.S. federal regulation issued in 1978. The definition reaches the full range of assessment techniques, through "informal or casual interviews and unscored application forms" 5.
A portfolio review that advances one candidate and drops another sits inside that definition, whether or not anybody wrote down a score. Ask the same questions of everyone who sent an artifact, keep the answers, and confirm the rules in your own jurisdiction with employment counsel. None of this is legal advice.
Where a prompt portfolio belongs in the loop
Before the exercise, and never instead of it. A prompt library is a strong agenda for a twenty-minute conversation and a weak substitute for watching somebody work. Use the folder to choose which two prompts to talk about, then run whatever exercise the loop already runs, on the same brief every candidate gets, including the ones who sent nothing.
Sometimes the library is the job. Somebody who maintained an internal prompt library for a two-hundred-person team did exactly the work the packet shows, and a fresh exercise can spend an hour re-answering what twenty minutes on that library already established.
The failure story is also coachable now, which blunts the best question on this page. Telling a real catch from an easy one is a separate problem with its own tells.
Candidates who sent nothing are the other half of the fairness question. Plenty of people doing serious AI work cannot show any of it, because an employer data policy owns the files.
A round that rewards the packet is partly scoring who was allowed to keep their work.
The folder does the same job a credential does, and it fails in the same place. A prompt engineering certificate against a work sample is the identical choice with a different artifact: one is a claim about capability, the other puts the candidate in front of a task.
Then open the folder again. Three custom GPTs and a prompt library, read cold, still say nothing about the person who sent them. Read after twenty minutes spent on what each one replaced, who else ran it, and the morning one of them stopped working, the same folder reads as a year of somebody's job, a personal shortcut, or a list of bookmarks. The files did not change. The reading did.
Common questions
Is a prompt library a work sample?
No. A work sample is a task the employer sets, performed under conditions the employer controls, and graded against something written before the submission arrived. A prompt library is a set of files the candidate chose, produced over an unknown period, with unknown help. It can point to what somebody says they worked on, and it settles nothing about how they work now. Keep it as material for a conversation, and keep whatever exercise the loop already runs, given to everyone on the same brief.
Do three custom GPTs show engineering ability?
Not by themselves. Building one can take no more than writing an instruction block, attaching a few files and naming the result, which is real configuration work and not model development. Three questions separate the cases: whether anybody other than the author used it, what it did when somebody asked for something outside its brief, and what got changed after the first week. A candidate who ran one in front of real users answers all three in specifics. A candidate who built three last month answers about intent.
What if a confidentiality agreement blocks the actual library?
Ask about the shape of the library instead of its contents. Which recurring task it addressed, how many people used it, how often it broke, who maintained it after the candidate left: none of that requires a file or a client name. Plenty of people doing serious AI work inside a company can show none of it, so a round that only rewards shareable files scores who was allowed to keep their work rather than who does it well. Offer the same exercise anyone else would get.
How many prompts should you actually read?
Two or three, chosen by the candidate as the ones that mattered most. Reading forty adds volume and no information, because the qualities that separate an assembled library from a maintained one do not live in the text of any single prompt. Ask which three they would keep if they could keep only three, then spend the time on what each one replaced and what happened when it broke. The choosing is informative by itself, and it costs a minute.
What does a strong answer about a prompt that stopped working sound like?
A strong account of a broken prompt carries a date, a symptom, a discovery route and a fix. Something in this shape: a model update in the spring made the summarizer invent section headings, the reviewer downstream caught it on the second batch, the fix was moving the section list into the input rather than asking for it, and two prompts built on the same pattern were changed at the same time. Notice how much of that is about the person who found the problem. A weak answer names no route and no consequence.
Should a prompt portfolio replace the take-home?
No, and it usually cannot, since the candidate chose the material and set the conditions. What a portfolio can do is aim the conversation: run the exercise the loop already uses, and spend the twenty minutes on the candidate's own choices instead of on generic questions. Candidates who sent no portfolio get the same exercise and the same twenty minutes, which is the part that keeps the round comparable.
References
- 1. Identifying and Improving Disability Bias in GPT-Based Resume Screening ✓ arxiv.org Supports the claim that configuring a custom GPT with instructions moves a model's output without stabilizing it: the instructed custom GPT ranked the enhanced CV first in 37 of 70 trials against plain GPT-4's 15, and still ranked the strictly stronger CV second about half the time, on a design of one CV, one job description and ten trials per condition.
- 2. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (arXiv:2507.09089) ✓ arxiv.org Supports the gap between belief and measurement in a candidate's own account of what a prompt saved: 16 developers, 246 tasks, 19% slower with AI tools, after forecasting a 24% speedup and still estimating a 20% speedup afterwards.
- 3. Reimagining Assessment in the Age of Generative AI: Lessons from Open-Book Exams with ChatGPT ✓ arxiv.org Supports asking for the working rather than the finished prompt: students submitted interaction transcripts alongside solutions, the strongest evidence of reasoning appeared where they evaluated incorrect or incomplete AI responses, and the author concluded that a correct final answer may no longer be sufficient evidence of comprehension. Cited as a qualitative single-course study, not a validated grading method.
- 4. Update on Misrepresentation of Research Publications Among Orthopaedic Surgery Residency Applicants ✓ ebi.ac.uk Supports the claim that a checkable field, rather than a warning, is what moves measured misrepresentation: 13 misrepresented citations of 1,100 in the 2016-2017 cycle (1.18%) against 18.0% in 1999 and 20.6% in 2007 at the same program, with the PubMed identifier named by the authors as their hypothesis for the fall.
- 5. 29 CFR Part 1607 - Uniform Guidelines on Employee Selection Procedures (1978), sections 1607.16(Q) and 1607.3(A) ✓ govinfo.gov Supports the claim that a portfolio review used as a basis for an employment decision is a selection procedure: the Guidelines define one as any measure, combination of measures, or procedure used as a basis for any employment decision, reaching the full range of assessment techniques through informal or casual interviews and unscored application forms.
5 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.