Screening
The Cold Email Stopped Sorting Sellers. Ask What They Disqualified.
Stop asking sales candidates for a writing sample. Drafting a cold email costs one prompt now, so the sample no longer separates candidates the way it did. Ask instead for three things that have to come from the candidate's own work: one account they disqualified and the sentence that ended it, one lost deal and the question they would ask earlier next time, and the last drafted message they killed before it went out. Ask every candidate the same three, in the same order.
The takeThe writing sample was never a writing test. It priced a candidate's willingness to do a boring thing well, and that price collapsed. What replaces it should be judged on appetite for subtraction: the account cut, the paragraph cut, the message that died in drafts. Sellers who work well with these tools generate far more than they send and are hard on the difference. A candidate who cannot name anything they deleted from recent work may be shipping whatever arrives, which is a habit the tool makes expensive.
Where Olive fits
Open a role and see what the work shows
Sales is not among Olive's twelve live occupation banks today, and the marketing bank is grounded in market research analysts' work, not in selling. What Olive returns on every bank is six findings from a 40-to-60-minute assignment worked alongside an AI assistant, each anchored to a timestamped excerpt, with the candidate granted the same report the employer reads.
Assess your shortlistWhat was the cold email ever sorting for?
Effort, mostly. A cold email asked a candidate to research an account, pick an angle, and spend real time proving they would do the unglamorous part of the job. The artifact was never the point; the willingness to produce it was.
Drafting that artifact now costs one prompt, so the sample still arrives on schedule and the thing it stood in for does not.
One measured result helps explain why the samples stopped sorting. In a pre-registered 2023 experiment, 444 college-educated professionals wrote occupation-specific press releases, short reports and analysis plans, half of them with ChatGPT 1.
The assisted half finished 37 percent faster, and their work was graded 0.45 standard deviations higher, with the largest gains going to the weakest writers 1. Those were single tasks, done once, online, for pay, with no colleagues and no revision cycle.
A test that sorts people works by spreading them out. Assistance lifts the bottom hardest.
But a narrower spread is no evidence that a particular sample was drafted by a model, and hunting for which ones were wastes the time a screen has. Non-expert readers asked to tell GPT-3 text from human writing across stories, news articles and recipes performed at chance, and three quick training methods lifted them only to about 55 percent 2.
The study ran in 2021, on crowdworkers rather than sales leaders, over short passages in three domains. Models have improved since, which probably moves unaided judgment the wrong way, though the study did not measure that.
A cover letter can be replaced elsewhere on the application, which is why what to ask for on the application instead of a cover letter is its own question. Outreach has no such exit.
Writing to strangers is the job, so the replacement has to come out of the sales process.
Ask for the account they disqualified
Ask for one account they walked away from, and the sentence that ended it. A model will draft a discovery call, a follow-up and a mutual action plan on request. What it cannot hand you is an account that existed, a date it died on, and a consequence the candidate carried into the next quarter. Disqualification is the cheapest of those to ask for and the hardest to perform.
Three questions, in this order:
1. The disqualification. Which account did you walk away from, and what was said that ended it? 2. The loss. Which deal slipped, and what would you ask earlier next time? 3. The kill. What is the last message a model drafted for you that you did not send, and why?
Grade the first on specificity, never on the outcome. Strong answers name what they heard: no budget holder in the room after two calls, a champion who could not say what the purchase solved. Weak answers name a category of bad-fit customer and stay there.
The second question tests sequencing. Listen for a question that moves earlier in the process. A resolution to try harder, or to qualify better, is the same sentence with one verb swapped.
The third question reaches the tool directly.
Someone who runs drafts through a model every morning has a recent kill and can say what tipped it. A claim about the prospect's roadmap that no public source supported. A familiarity the buyer had not earned.
The same reading applies to any story a candidate volunteers about a model getting something wrong. The grade belongs to how hard the catch was, and telling a real catch from an easy one is most of the work.
A kill the candidate chose is stronger evidence than a mistake the candidate noticed, because the decision to delete was theirs.
The three questions leave out which tools they use, because a list of tool names says nothing about how any of them was used. What a strong answer about refusing to delegate sounds like is closer to what you want: the boundary worth finding is where a seller stops handing work over.
Write the outreach yourself, then ask what they would cut
Bring the draft to the call instead of requesting one. Before the interview, generate three cold emails to a real prospect in your own market, then hand them over on a screen share and ask the candidate what they would cut and why. The exercise is short, the same for everyone, and it puts the artifact where it belongs: in front of a person who has to defend a deletion out loud.
Deletion is the part of the job that never got cheaper.
Listen for cuts that cost something. Strong candidates cut the flattering opening line, the invented detail about the prospect's quarter, and the claim they could not defend if the buyer wrote back. Weak candidates rewrite for tone and hand back the same message, shorter.
One 2026 study points the same way from a classroom. Engineering students allowed to use ChatGPT on take-home open-book exams had to submit their chat transcripts alongside their answers.
The author reports that some of the strongest evidence of reasoning appeared where a student evaluated an incorrect or incomplete ChatGPT response, and concludes that, where the tool is permitted, correctness of final answers alone may no longer be sufficient evidence of comprehension 5.
One course, qualitative, no control group and no comparison of grades.
At least two things do not transfer to hiring: a transcript can be curated afterwards, and asking for one is a disclosure demand a candidate may have reason to refuse.
In sales, the line nobody cut still goes out: the document reaches a stranger under your company's name.
Ask who owned the opt-out
Ask who owned the opt-out on the last sequence they ran. A seller who has sent automated outreach at volume sent it under rules somebody had to own, and the answer says whether they knew who. The question is not a legal quiz.
You are finding out whether the volume they report ever came with a consequence attached.
The core email rules are short enough to hold in your head while they answer. The federal floor for commercial email in the United States is the CAN-SPAM Act of 2003.
Under 15 U.S.C. 7704, a commercial message may not carry materially false or materially misleading header information 4. Nor may it carry a subject heading that, on the sender's actual knowledge or on knowledge fairly implied from objective circumstances, would be likely to mislead a recipient acting reasonably about a material fact regarding the contents or subject matter of the message 4.
Two of the requirements are clerical: identify the message as an advertisement or solicitation, and carry a valid physical postal address 4.
The requirement a sequencing tool has to keep honoring after the send is the opt-out. Section 7704(a)(3) requires a functioning return email address or other internet-based mechanism for opt-out requests, clearly and conspicuously displayed, that keeps receiving them for no less than 30 days after the original went out 4. Sending a message that falls within the scope of a recipient's opt-out request, more than 10 business days after the request arrives, is unlawful 4.
Section 7704 covers commercial email only, and calls and texts are a separate question.
The specifics move, and they are a question for counsel.
What the question tests is ownership. Nobody needs the section number, and producing one earns nothing on its own.
Grade it the way you graded the disqualification: on what the candidate can name. A seller who says the sequencing tool handled the unsubscribes has told you where the knowledge lived, which is usable.
Better is the one who can also say who would have heard about it first if a list went out wrong.
The weak answer is a policy sentence with nobody in it. The company follows the rules. The tool is compliant. Nothing in that names a person who would have picked up the phone.
A candidate who has never thought about it, while quoting outreach volume as an achievement, has told you something else. The volume was somebody else's risk to carry.
Run the same three questions on every candidate
All three questions go to every candidate in the same order, rated on a scale written before the first call. Structure is the part of this with evidence behind it. In a 2022 re-analysis of the selection literature, structured interviews come in at a mean validity of .42 against supervisor ratings of job performance and unstructured ones at .19, a gap wider than the distance between most whole methods 3.
Structured there means a format: the same questions, a common rating scale. Not a script and not a product. None of the underlying studies involved a question about how anyone uses AI, and most tested people already doing the job.
Write each point of the scale as something a candidate could actually say, before you use it. On the disqualification question, four points run from no example, to a category of bad-fit customer, to a specific account with a reason, to that account plus the sentence that ended it. An account described without its name counts as specific.
A twenty-minute screen does not hold three questions and a draft. If only one fits, pick one load-bearing claim before the call and stay on it; the disqualification is the one that travels best.
A seller's best example may belong to an employer who would rather they said nothing. A candidate under a confidentiality agreement answers thinner than one never asked to sign.
Structure can at least stop the thinness from reading as evasion.
Keep the writing sample if it costs nothing. Move it to the end, after the three questions, and read it for one thing: whether it names an account the candidate could still describe out loud and a reference could confirm.
The sample stopped being a test the month drafting got free. As a receipt against what they told you, it still holds.
Common questions
What should you ask a sales candidate for instead of a writing sample?
Three things, asked of every candidate in the same order. One account they disqualified, with the sentence that ended it. One deal that slipped, with the question they would ask earlier next time. One message a model drafted for them that they chose not to send, with the reason. Each requires an account that existed, a date and a consequence, which is exactly what a drafting tool cannot supply. Rate the answers on a scale written before the first call, so two interviewers reach the same number on the same answer.
Can you tell whether a cold email was written by AI?
Reliably, no. Non-expert readers asked to separate GPT-3 text from human writing across stories, news articles and recipes performed at chance, and three quick training methods lifted accuracy only to about 55 percent. That study used crowdworkers in 2021 rather than sales leaders reading outreach in their own market, and models have improved since, which probably makes unaided judgment harder still, though the study did not measure that. Treat the question as unanswerable and spend the time on the account behind the email instead: which one, what was said, what happened next.
Is it fair to ask a candidate about a deal they lost?
Yes, if you ask everyone and rate the answers the same way. A loss is a past-behavior question about the candidate's own work, and sellers expect it. Two cautions. Grade the reasoning, because a lost deal is often the correct outcome and a candidate who never loses one is either new or not counting. And expect thinner answers from anyone under a confidentiality agreement, which is a constraint on the evidence rather than a signal about the person.
Should a sales screening exercise ban AI?
Not where the job permits the tool. A ban tests something the role never asks for, and it hands the round to whoever is best at working without an assistant they will use on day one. The stronger design permits the tool openly and grades the deletion: generate the outreach yourself, hand it over, and ask what the candidate would cut and why. Say in the invitation that the draft is provided and that the cutting is what gets rated, so nobody arrives having prepared the wrong thing.
Do you still need a writing sample for a sales role?
Only as a receipt, and only at the end. Once you have asked about the disqualified account, the lost deal and the killed draft, a sample is useful for one narrow check: whether anything in it lines up with an account the candidate described out loud. A polished document that names nothing traceable tells you little. A rough one built around a real account the candidate can still walk through, and a reference could confirm, tells you more than any amount of polish. Do not rank candidates on the artifact.
References
- 1. Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence (working paper, not peer reviewed) ✓ economics.mit.edu Supports the claim that assisted writing compresses the spread a writing sample exists to measure: 444 college-educated professionals on occupation-specific writing tasks, the ChatGPT half 10 minutes (37%) faster than a control averaging 27 minutes, grades 0.45 standard deviations higher, largest gains to the weakest writers.
- 2. All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text ✓ aclanthology.org Supports the claim that reading a sample to work out whether a model wrote it is a poor use of screening time: untrained evaluators separated GPT-3 text from human writing at random chance across stories, news articles and recipes, and three training methods lifted accuracy only to about 55%.
- 3. Revisiting Meta-Analytic Estimates of Validity in Personnel Selection: Addressing Systematic Overcorrection for Restriction of Range ✓ gwern.net Supports the claim that asking the same three questions of everyone on a common rating scale is the part of this with evidence behind it: mean validity estimates of .42 for structured interviews against .19 for unstructured ones.
- 4. 15 U.S. Code § 7704 - Other protections for users of commercial electronic mail ✓ law.cornell.edu Supports the outreach compliance floor a sales candidate should be able to name: (a)(1) materially false or misleading header information is unlawful; (a)(2) deceptive subject headings; (a)(3)(A) a clearly and conspicuously displayed functioning return address or other internet-based mechanism that remains capable of receiving messages for no less than 30 days; (a)(4)(A)(i) no transmission more than 10 business days after receipt of an opt-out request; (a)(5) clear and conspicuous identification as an advertisement or solicitation and a valid physical postal address of the sender.
- 5. Reimagining Assessment in the Age of Generative AI: Lessons from Open-Book Exams with ChatGPT ✓ arxiv.org Supports the claim that a finished artifact is no longer the evidence and the judging has to be made visible: students submitted ChatGPT transcripts with their exam solutions, some of the strongest evidence of reasoning appeared where a student evaluated an incorrect or incomplete response, and the author concludes correctness of final answers alone may no longer provide sufficient evidence of comprehension. Qualitative, one course, no control group.
5 sources, numbered by first appearance. Every one was opened and checked against the claim it carries. How Olive sources claims
General guidance for hiring teams. What works at one company and one volume may not transfer to yours.
Olive assesses how a person works with AI. It does not detect AI-written documents, and it never produces a score, a ranking, or a match percentage for a person. Candidates read the same report the employer reads.