The harness around AI: why the same model works in one organisation and not in another
The same AI model delivers reliable work in one organisation and a daily series of surprises in another. The difference rarely lies in the model alone. It lies in what surrounds it: agreements, access and a check that does not come from the model itself, and that is a management question more than a technical one.
Picture this: in the same department, two people work with the same AI assistant, which has access to their files and systems. One has it prepare a quote, summarise a report and check a schedule, and saves time with it every week. The other explains to it every morning how the company works, which client is sensitive and which figures must not simply change, and then checks everything again herself. After three weeks, the first finds it indispensable and the second finds it overrated. Both are right about their own experience, and both are mistaken about the cause.
The same model, two experiences
A talent without an onboarding pack
A language model resembles an exceptionally talented newcomer walking in on their first day. They know a lot, learn fast and do not know the house. What makes them productive is what gets organised around them: the agreements, the keys they do and do not get, and someone who reviews their work. In the AI world that whole is called the harness, and "harness" is common parlance there. Anthropic uses the word for its development kit for agents, OpenAI devoted a long piece to it, and researchers at Google describe it as everything a model uses to carry out its task: instructions, memory, skills, tools and the orchestration of that whole.
Where that whole is missing, a person takes over the role. The second colleague from the opening scene does exactly that. Every day, by hand, she supplies the context, the rules and the checks that are recorded nowhere else.
The harness: everything around the model
What a better environment delivers
In a recent study, not yet peer reviewed, researchers from Google Cloud among others had a model perform a professional task again, with the same model but an adjusted way of working that a second AI system had proposed. The score went from 43 to 78 out of a hundred. That took more than one extra rule to check its own work. Among other things, the model had to trace every figure back to its source and reopen and review its own files, and it was given domain-specific working methods. According to the authors, the errors of the first attempt were procedural rather than a lack of knowledge. It remains one task, chosen by the researchers themselves as an illustration. Across all test tasks together, the gains were much smaller.
An independent study with office tasks, also a preprint, points the same way: the same model scores clearly differently in different off-the-shelf environments, and which environment works best differs per model. According to that research, a universally best harness does not exist. OpenAI wrote about an experiment of its own that the work progressed slowly at first, "not because Codex was incapable, but because the environment was underspecified".
Same model, different way of working
Three choices that are not IT choices
What goes into such a harness sounds technical, but every organisation knows these questions from onboarding a new employee.
First, the agreements. Agents such as Claude Code read a file with standing instructions at every session, such as CLAUDE.md or the open format AGENTS.md. Whoever writes down the way of working properly once does not have to repeat it every morning. More is not better here. OpenAI saw a large instruction file fail and got better results with a short table of contents that points to deeper documentation. An agreement does remain a request. Anthropic writes that Claude treats such files "as context, not enforced configuration". What truly must not happen requires a technical boundary, although choosing what must not happen remains a management decision.
Three choices that are not IT choices
Then, access. Anthropic recommends the smallest possible set of tools, so that an agent does not get confused, and OpenAI advises rating every tool for risk, with human oversight for actions that are sensitive, irreversible or high-stakes. How it goes wrong without that boundary became clear in July 2025. During a public experiment by SaaStr founder Jason Lemkin, a Replit agent wiped data from a production database, despite an agreed code freeze. Replit CEO Amjad Masad called it "Unacceptable and should never be possible" and announced a separation between development and production data. Replit pointed out that it has backups that can restore a project in one click. Even so, it shows what an agreement is worth without a lock.
Finally, the checks. Asking a model to proofread its own work usually yields little in research without an outside test. Research by Google DeepMind found that language models struggle to improve their reasoning on their own without external feedback, and sometimes even get worse. A survey study reached a similar conclusion: self-correction works mainly when there is reliable feedback from outside. Models trained on it at scale do better, but the reliable gain comes from outside. Anthropic saw its agents confidently praise their own work, even when it was mediocre. A check that works tests against something outside the model: a source, a test, a checklist or a second reviewer. The intervention in the Google study was of that kind.
A lock on the door, a test from outside
“Organisations have always written their rules for people, and people silently fill in the gaps. A machine does not, and so for the first time we see how much was never said.”
The model does matter
It does not follow that the model is no longer the bottleneck. That would go too far. New model generations still make the big leaps: METR finds that the tasks models can complete on their own have been getting longer at a rapid pace since 2019, especially for software and research tasks. In the Google study, the strongest model tested gained the least from a better harness, and the stronger model stayed ahead when both got one. The makers of a minimal coding agent report that it achieves high scores on a well-known coding benchmark with strong models.
Add to that an older argument. In 2019, AI researcher Rich Sutton described as the "bitter lesson" that built-in human knowledge helps in the short term and loses in the long term to methods that scale with computing power. Applied to harnesses, that means: what a harness catches for the model today, a next model may be able to do itself. Anthropic writes something related: every part of a harness contains an assumption about what the model cannot do alone, and those assumptions can go stale quickly as models get better. What is necessary today may be ballast with the next model.
The model matters, the harness shifts
Whoever writes about harnesses often has a stake in them: Anthropic and OpenAI sell them, Google publishes its own framework for them. The independent study on office tasks and the survey on self-correction do point in the same direction.
What remains is a sharper division. The model determines what is possible. The harness determines how much of that an organisation gets back reliably and repeatably. The first is bought in, the second an organisation has to arrange itself. According to Anthropic, the space for interesting harnesses does not shrink as models get better, it moves. The technical part may therefore be replaceable. The questions of what is allowed, who approves and what things are checked against do not disappear with a new model. Almost all measurable evidence comes from test environments, mostly with coding and office tasks. The translation to real organisations therefore remains a line of reasoning.
What you buy and what you arrange yourself
Who does not build the harness
Back to the department. The first colleague does not work with a better model. She has written down somewhere how the work goes, given the assistant no more access than needed and found a fixed way to check before anything goes out the door. The second does the same work, only every day again and by hand. In many organisations, AI looks like this today: a strong newcomer, and a colleague who onboards it again every morning.