I don't want the LLM to be the smartest part of my system.
I want it to be the smallest part.
That sounds like an insult to the model. It isn't. The model is often the only reason the product is worth building. Ranking a messy candidate pool, explaining an API route, interpreting a customer reply: those are judgment problems. Deterministic software is bad at them. Language models are good at them.
The argument is about responsibility, not intelligence.
The more of the system you hand to an LLM, the more of the system becomes probabilistic. Probabilistic systems are harder to reason about, harder to test, harder to debug, and harder to trust. If "probably correct" is not acceptable, the model should not own that step.
A good production system should be mostly boring software.
The interesting part might be one model call. Everything around it should make that call safe, bounded, observable, replaceable, and useful.
The LLM is not the architecture
I keep seeing diagrams where the model sits in the middle and everything else talks to it.
The model becomes the router. Then the planner. Then the parser. Then the classifier. Then the database interface. Then the business logic. Then the tool orchestrator. Then the writer. Then the decision-maker.
Pretty soon the system is:
Everything → LLM → Everything
This is seductive for obvious reasons. The model is flexible. It can interpret messy input. It can call tools. It can make decisions. It makes a demo look like you skipped six months of product design.
Flexibility is not a boundary.
Every extra responsibility you give the model is another place the system can be "kind of right." Kind of right is fine in a playground. Kind of right is how you email the wrong founder, document an endpoint that does not exist, or refund someone because a transcript sounded angry.
The common anti-pattern is more specific than "put GPT in the middle":
The point is not that this architecture can never work. It is that most production systems do not need to give the model that much authority. You already know most of the steps. You are using a language model as a substitute for writing them down.
The better pattern is narrower:
Software → LLM → Software
The software owns the system. The LLM handles the uncertainty. That is a smaller job. It is also the only job the model is uniquely good at.
What should NEVER be an LLM?
Some of this should not even be a debate.
A prompt is not an ACL. If the model is deciding who is signed in, or who can see a workspace, you do not have authentication and authorization. You have vibes with a login screen.
The same is true of the rest of the machinery production systems are actually made of. Database filtering. Business invariants. Workflow state. Retries. Idempotency. Scheduling. If you already know the rule, write the rule. Asking a model whether a row is overdue, whether this domain has already been contacted, or whether the cron should fire again is not architecture. It is avoiding a WHERE clause.
Money is the extreme version of the same mistake. Fuck no. "Probably sent the payout" is not a payment system.
JSON is almost in that bucket too. You already have a parser. Do not ask a model to discover what a document means when the document already has a schema.
The useful work starts where that certainty runs out. Two messy pieces of text that might mean the same thing: maybe. Two hundred candidates against a job description: that is where the model can earn its keep.
Don't use probabilistic software for problems where you already know the answer can be deterministic.
Putting those problems in a prompt does not make them smarter. It makes them deniable. The model should not own side effects, permissions, or transactional logic. If the answer has to be the same every time, write the code.
Deterministic software is underrated
A surprising amount of production engineering is the unglamorous work of making things certain. Status checks. Unique constraints. Parsers. State machines. The stuff that does not photograph well in a demo and is the reason the demo can be trusted.
If you can write this, the model has no job here:
if (invoice.status === "overdue") {
enqueueReminder(invoice.id);
}
The LLM should not replace that class of system. It should sit next to it.
This is the same instinct as picking a boring stack. Users do not pay you to look clever. They pay you for a system that behaves. The model is the novelty. The rest of the product should not try to compete with it for excitement.
The uncertainty boundary
Every AI system has an uncertainty boundary.
On one side: rules, schemas, queries, parsers, state machines, business logic, validation.
On the other: judgment, ambiguity, classification, ranking, interpretation, generation.
The mistake is letting the LLM cross that line and take responsibility for things the system already knows how to do.
Put the LLM exactly where certainty ends.
Not before.
Not everywhere.
Not nowhere.
Exactly there.
That diagram is the whole article, if you squint.
The model makes one judgment. One. The software decides what is true before that call, and what happens after it.
If you cannot point to the line, the model already owns too much.
A useful test: if this step failed, would you want to reproduce it with the same input every time? If yes, keep it in code. If the same input can honestly produce two different valid answers, and a human would also have to judge, that is the boundary.
Which candidate is the strongest match?
Does this customer reply indicate buying intent?
How should this API endpoint be explained?
Which explanation best fits this user's question?
Is this piece of content likely relevant to the user's intent?
Those are uncertain. They belong to the model.
"Is this user allowed to see this workspace?" is not.
Once you draw the boundary, the question changes. You stop asking where to put AI and start asking where certainty ran out.
DocPilot
DocPilot is the simplest version of this I have shipped.
The wrong architecture is:
Repository → LLM → "figure out the API" → documentation
That version worked on my test project. It hallucinated endpoints on everyone else's. The model was being asked to discover the system, parse the system, and narrate the system in one breath.
The better architecture is a smaller job for the model.
The model is not responsible for discovering everything. It is not responsible for deciding what the product considers an API route as a matter of truth. Extraction can veto a bad classification: if the file has no handler, skip, don't invent. I have already written about that split. The point here is the boundary, not the pipeline trivia.
OpenAPI and PDF uploads make the same point from the other direction. Those inputs go through deterministic parsers first. The model does not "discover" a spec that already has a spec.
The model has one important job: turn structured technical context into useful developer-facing explanation. That is enough. If generation fails, you still know which routes exist. If classification is noisy, extraction still refuses to document middleware.ts.
The outbound engine
The automated recruiting outreach engine looks more autonomous. Cron fires. Companies get discovered. Applicants get ranked. Emails get queued.
From the outside, that can look like an agent. Inside, it is a control loop I wrote.
The wrong architecture is:
Company → LLM → "figure out everything" → email
The better one:
The model is useful because "who is the best match?" is a judgment problem. Ranking a resume against a job description, or against the team a company already hired, is not a SQL query.
The model should not decide whether a lead has already been contacted. That is an exclusion list.
It should not decide whether an email should be sent twice. That is idempotency.
It should not decide whether the cron retries. That is the task runtime.
It should not decide whether a company exists in the database.
The LLM should rank. The system should operate.
Discovery, enrichment, skip conditions, fan-out, gates, and delivery are software. The model sits in a small box in the middle and returns a score. The orchestrator can fan that box out and still remain a control loop a human can read.
Pulse
Peergrowth Pulse has an AI copilot. I built it on purpose. This is not an argument against copilots.
It is an argument about what the copilot is allowed to own.
Pulse already owns campaign data, spend, leads, attribution, historical performance, and workspace context. The model should not retrieve that state by guessing. It should not invent CPL. It should not mutate a lead because a sentence sounded like a follow-up.
The product gives the model the relevant context. In Pulse that means the same analytics primitives the dashboard and reports already use, exposed as read-only tools, with numbers required to come from those tools. The uncertain question is:
Given what happened here, what is probably causing this change and what deserves attention?
That is judgment. The product remains responsible for the data, state, permissions, and actions. Syncing leads, capacity-matching attribution, and freezing a client report are not model problems.
I wrote about the interface half of this in I Don't Want Your AI Copilot. The architectural half is the same line: even when the interface is a conversation, the model is still a component. It is not the source of truth.
"But agents can do all of this"
They can. That is not the interesting question.
Agents are useful when the next action genuinely cannot be known ahead of time. A coding agent in an unfamiliar repo. Research across messy sources. Work where tool choice is the product.
If you already know the workflow:
discover → enrich → classify → rank → validate → act
then putting an autonomous agent in charge of choosing those steps often adds complexity without adding useful intelligence. A model choosing between known deterministic steps is not automatically a better architecture. Sometimes it is just a workflow with an LLM supervising the workflow.
This article is the next step of that argument.
If the agent shouldn't own the workflow, the LLM shouldn't own the rest of your application either.
The model does not become more trustworthy because you also let it call sendEmail. It becomes harder to debug. Authority is not a feature. It is blast radius.
What a small model call buys you
This architecture is not aesthetic. It is operational. Every benefit comes from the same line: certainty stays in code, judgment stays in one call.
You can change models without rewriting the application, because the application depends on a contract, not a personality. You can test the deterministic parts normally and evaluate the model at a narrow boundary. You know which stage failed, because classification, ranking, validation, and delivery are different logs. You don't spend tokens listing files you can walk or filtering rows you can query.
The model also has less authority. It cannot grant access, move money, or fire side effects except through code that already decided those things were allowed. Bad output can be rejected before it becomes an invoice or an email. If the model fails, the rest of the system can often continue: routes still exist, companies still get skipped, dashboards still show numbers. That is graceful degradation as a side effect of keeping the model small, not as a special AI technique.
A model call with a contract is an API. A transcript of a tool loop is not.
The model should be replaceable
If replacing GPT-4o with another model requires rewriting your application's business logic, you have coupled the product to the model too deeply.
The application should depend on a contract:
type MatchResult = {
score: number;
rationale: string;
redFlags: string[];
};
not:
"whatever GPT-4o happens to return today"
The prompt is implementation detail. The schema is the interface. I have already argued that at length. The reason it matters here is the uncertainty boundary: once the model only owns one judgment, that judgment can have a typed shape, and the orchestrator does not care which provider filled it.
You can rewrite the prompt. You can swap Gemini for GPT. You can fan the call out, retry it, or skip it. The rest of the product stays the rest of the product.
A model that owns auth, state, and side effects cannot be swapped. You would be swapping the application.
Code decides what is true
A useful mental model:
Code decides what is true.
The LLM decides what is ambiguous.
Code decides what happens next.
Code: "Is this user authorized?"
LLM: "Does this support message sound like an angry cancellation request?"
Code: "Should we refund them?"
The model can interpret. The system should enforce.
Interpretation is allowed to be probabilistic. Enforcement is not. Reverse the split and you get systems that feel magical in a demo and unaccountable in production. Nobody can tell you why the refund went out, only that the assistant was "pretty sure."
The LLM should be the dumbest part of your system
The title is not about making the model worse.
It is about making the system stronger than the model.
The goal is not a powerful LLM. The goal is a powerful system that uses an LLM where software runs out of certainty.
A good AI system should feel like:
Lots of boring software
+
One carefully bounded uncertain step
=
A useful AI product
Not:
One giant model call
+
a prayer
=
production
I don't want a genius in the middle of the diagram. I want a boring machine with one sharp judgment in it.
Don't make the LLM the system. Make it the smallest replaceable part of a system that already knows what it's doing.
Related: Why Most AI Products Don't Need Agents · Schema-First LLM Systems · Two-Step LLM Pipelines · I Don't Want Your AI Copilot · Graceful Degradation in Cron Pipelines · DocPilot case study · Automated Recruiting Outreach · Peergrowth Pulse