AI harness: everything you build around a language model

An AI harness (also called an LLM harness) is everything you organise around a language model to make it useful for a real business task: the context the model receives, the tools it may use, the orchestration between tasks and the feedback that lets the system learn from mistakes. The model itself has become interchangeable; the harness decides whether an AI application delivers value or quietly fizzles out.

This page gives you the overview: what a harness is, why it matters more than the choice of model, what our own harness looks like, where to start, and what it means for the EU AI Act. Each part points to a full article.

What is an AI harness?

An AI system in production has four moving parts. The model (Claude, GPT, Gemini or an open source variant) produces text. The context decides what the model knows for each task: documents, customer data, agreements, house style. The tools decide what the model can do besides talk: read email, draw up a quote, create a task. The orchestration decides how tasks are divided, who checks what, and what happens when something goes wrong.

The harness is everything except the model. Birgitta Böckeler of Thoughtworks captures it in the formula that now circulates across the field: agent = model + harness. The term was put on the map in early 2026 by Mitchell Hashimoto (co-founder of HashiCorp): every time an agent makes a mistake, you do not fix the mistake itself, but the environment around it, so the same mistake can no longer happen.

Read more: LLM harness, the new development in 2026 →

Why the harness matters more than the model

On most business tasks the large language models are converging. For summarising, writing, classifying and simple analysis they are all more than good enough, and you can switch models without losing quality, as long as your harness is ready for it. What the model does not know is how your business works. Poor context gives poor output, whichever model you pick. And one good answer is not a workflow: the difference between an AI that writes a tidy email and an AI that recognises a complaint, pulls up the file, drafts a reply and has a colleague approve it lies almost entirely in the harness.

What our own harness looks like

We treat ourselves as our first client. Our own harness consists of three context layers (one of them deliberately without AI access), eighteen specialised agents each with a limited set of tools, two agent teams with three main pipelines, and a feedback system with three levels. In a separate article we describe how it was built, where we got stuck, and what of it is useful for other businesses.

Read more: how we set up our own AI harness →

The context layer: start with your knowledge

Of the four parts, context pays off fastest. An AI knowledge assistant (an assistant that answers questions based on your own documents, agreements and processes) is therefore the logical first step of a harness for many organisations: you build the context layer that every later application needs, and your team notices the difference straight away.

Read more: an AI knowledge assistant as the first step of your harness →

What a harness handles for the EU AI Act

From August 2026 most obligations of the EU AI Act apply, including for businesses that only use AI. Much of what the law asks (knowing what data a system sees, human oversight of sensitive steps, being able to trace what happened) overlaps with what a well-designed harness already does. A harness is not proof of compliance, but whoever has their harness in order has most of the homework already done.

Read more: what a good AI harness already handles for the EU AI Act →

When you do not need a harness

For one-off tasks (translating an email, shortening a text, brainstorming) a regular ChatGPT or Claude session is fine. A harness becomes important once a task recurs, consists of several steps, is used by several people, or once mistakes cost money or reputation. At that point an ad hoc approach falls apart fast.

How we can help

If you want to know where in your organisation a harness pays off first, the AI scan maps that out: one process, concrete advice, a realistic estimate.

If you then want to actually build it, from context layer to working agents with human oversight, we do that together in AI in Business.

Book a no-obligation call and we will look together at where the first gains are for your organisation.

Book a call Take the AI scan

Frequently asked questions

What is an AI harness?

Everything you build around a language model to make it useful for a business task: context, tools, orchestration and feedback loops. The model provides the language ability; the harness makes sure that ability works within your processes.

Is an AI harness the same as an AI agent?

An agent is the combination of both: agent = model + harness. You can buy or rent the model; the harness is the part you set up yourself and that decides how well the agent works.

So does the choice of model not matter at all?

For most business tasks, little. The large models have become interchangeably good and a good harness makes switching easy. The debate over which model deserves less time than the question of which context and tools you have in order.

Where do you start with a harness?

With one recurring process, and usually with the context layer: make sure the information AI needs is findable and structured. A knowledge assistant on your own documentation is a good first step.

Does a harness help with the EU AI Act?

To a large extent. Logging, human oversight and control over data are parts of both a good harness and the law. Formal obligations remain; a harness makes demonstrating compliance much easier.