Every new AI chat starts as a stranger. It does not know your company, your rules or what you did yesterday. So you explain. Then you explain it all again in the next chat.
At First AI Movers, the roadmap (the company’s list of planned work) put a price on that. Each new conversation cost the founder 20 to 30 minutes of explaining. The roadmap summed it up in four words: “Compounding loss across a week.”
The plan was a page. Paste it into any AI assistant as the first message, and the stranger would know enough to help. The roadmap also said why nobody should just trust that plan: “Documents are theory until proven.”
So on a Sunday in May, the page was put to the test. Three assistants got it, then the same question. Two of them answered the question. The third did not. It asked for the files instead.
The test results called that third one “the strongest signal of all three.”
Why would a test single out the one that did not answer?
Who’s who
- The Operator The founder. He ran all three test chats.
- ChatGPT Project A paid ChatGPT workspace with a connector: a key that lets it read the company’s shared folder.
- Claude Cowork A Claude workspace that could also read the folder.
- Plain Claude.ai An ordinary new Claude chat, with no key and no files.
The job and the question
The company’s org chart had a name for the job: “Technical team lead.” That meant an AI that turns the founder’s wishes into work orders and checks facts, but does not change the code itself.
The page came in three versions, one shaped for each assistant. The test plan was strict. Open a brand-new chat. Paste in the page. Wait for the reply. Then send this exact question:
Bootstrap against the current agent-toolkit repo. Tell me: 1. What is the current ROADMAP position? 2. Which sprint should we work on next and why? 3. What files did you verify? 4. Give me one Claude Code meta-prompt for the next concrete deliverable. Use certainty labels.
In plain words: read our shared folder. Tell me where the work stands, which small task (a “sprint”) comes next and why, and which files you checked. Then write the work order for the next job. That is the “meta-prompt”: instructions one AI writes for another AI to carry out. Tag every claim with how sure you are: verified, inferred, unresolved or estimated.
Then close the chat right away, the plan said, so the result would show only what the page itself made possible.
The question had a twist built in. On the roadmap that day, task B.3 was marked in progress. Task B.3 was this test. So the right answer to the first question was: in the middle of the very test you are taking. And any assistant that made things up failed. One file, task name or version code that did not exist was enough.
The results list the three in this order. They do not say which chat ran first.
ChatGPT Project: by the book
ChatGPT had its key, and it tested the key before leaning on it. Before it claimed anything that mattered, it confirmed it could really reach the folder.
Then it read. It listed the files it had checked, each with its exact version code. It also confirmed which version of the folder it was reading: the one saved when this very test was set up.
It found the big goal: letting the founder pick up his work in any assistant. The goal’s first two tasks were done. The third was in progress. The third was the test it was taking.
Then it did the job, and only the job. It treated the request as part of the current task, and as checking work only. It wrote the work order instead of doing the work itself.
Verdict: pass.
Claude Cowork: one step ahead
Cowork read five files. Among them were the roadmap, the org chart and the plan for this test.
That last one is worth a pause. Cowork was reading the instructions for the test it was in. It worked out that this very chat was its own row in that plan.
It saw that the test was the task in progress. It even wrote, in plain sentences, that finishing the test was strictly the next step.
Then it wrote the work order for something else.
The order was for task E.1, a first draft of a page for tracking experimental features. E.1 was a real task on the real roadmap. It just was not next. The results named the rule Cowork broke: “current sprint should be completed before jumping ahead unless the operator explicitly approves reordering.”
Verdict: “partial pass,” with one defect.
What the results said next is worth noticing. They put the defect in the page, not in Cowork: “No re-test required — the defect is template wording, not Cowork’s reading or verification behavior.” (The template is the page.)
So the page was fixed, in the same change that recorded the verdicts. Cowork’s version got a new heading, “Sprint discipline:”, and a new rule: “When a current sprint is already in progress, complete or close that sprint before proposing a different next sprint.” One line even named the task Cowork had jumped to: “Do not jump to E.1, C.2, or any other sprint while a portability test is … in progress.”
And E.1? It stayed not started, “queued for [the Operator]’s explicit go.”
Keep that line in mind.
Plain Claude.ai: the one with nothing
The third assistant had no key. No workspace, no connector, no files. Its page told it so, in capital letters: “You do NOT have a GitHub connector or repo access in this chat.”
Think about what that meant. The question asked where the work stood, what came next and which files had been checked. Every answer sat in a folder this assistant could not open. And making things up meant failing.
Its page had planned for this. The folder’s guide called this version “Worst case; carries the most context.” Its second rule said what to do at the wall: “Halt and ask [the Operator] to paste the required files if you need them. Do not invent file contents.” If a fact that mattered could not be read, the rule said to tag it “[UNRESOLVED]” and ask for the file. The page even explained itself: “Plain claude.ai gets nothing — so the prompt halts explicitly when it would otherwise have to invent.”
So it halted. It checked what it had to work with, and found nothing. It asked the Operator for the folder’s web address, links to the files, or the files pasted in. It did not say where the work stood. It did not write a work order. It tagged every claim that mattered “[UNRESOLVED]”: not known, cannot check.
On paper, one way to pass was to name the test as the task in progress. Plain Claude never named it. It had no way to see it.
The results passed it anyway, and gave its kind of pass a name: “pass-by-refusal.” Then came the line this story started with, in full: “This is the strongest signal of all three: with no repo access, the template’s halt-rather-than-invent rule is exactly what should fire.”
In plain words: the page said to stop rather than make things up, and this was the exact moment for that rule. The other two showed the page could send a reader to the right files. This one showed what the page did for a reader with nothing but the page. The rule fired. The assistant stopped and asked. The results called that “the expected and desirable behavior.”
Six minutes
The test was closed with this verdict: “Sprint B.3 passes with one template fix applied.”
Six minutes later, the roadmap marked E.1 done. That was the task Cowork had jumped to, the one waiting for the Operator’s go. The record does not say whether he gave it in between, or how. It shows only the order. The test was finished first. E.1 came next.
Cowork had named the right job. It had named it one step too soon.
What any founder can take from this
- Test your briefing on a stranger. Give a fresh helper your onboarding page and one fixed question, and add
nothing. What they get wrong tells you what the page leaves out.
- Say what to do when the answer is out of reach. Write it down: stop, say so, ask for what is missing. Then
count that as a pass.
- When a helper jumps ahead, check the instructions first. Here, the fix was a few lines in the page, and no
re-test was needed.
- Keep the order of work in the owner’s hands. Finish the job in progress unless the owner says otherwise.
Next time: “The Memory Came With Tic-Tac-Toe.”
How this article was made: This story is based on the company’s own records from 2 and 3 May 2026: the roadmap, the org chart, the three briefing pages and their guide, the test plan and the written test results. Those records summarize what each assistant did; they do not hold the chats themselves, so no assistant is quoted, and every quoted line comes from the pages, the plan or the results. The chapter headings and the pictures of a stranger and a key are the writer’s. The chapters follow the order of the results; the order the chats ran in is not recorded. In quotes, the founder’s name is replaced with [the Operator], and one status symbol is cut and marked with an ellipsis. Corrections are welcome and will be dated.