Build vs Buy
11 minute read
This course built a gatekeeper, guardrails, memory governance, hallucination checks and an evaluation harness. Should a production team build all of that? Mostly not. Some of it, absolutely. These notes give the tools to tell which: a per-component framing with four options (build, adopt, buy, partner), the arithmetic of ROI, payback, NPV, total cost of ownership and break-even volume, and the non-financial factors that often decide the matter.
By the end of these notes you should be able to:
- frame build-versus-buy per component rather than per agent;
- compute ROI, payback, NPV and a break-even volume, and interpret them;
- explain why the falling price of model capability changes what is worth building;
- recommend what to build, adopt and buy for an agent, and justify it.
“Should we build or buy an AI agent?” is the wrong grain. An agent is a dozen components, and the right answer differs per component. And there are four options, not two:
| Option | Means | The catch |
|---|---|---|
| Build | Write and own it | You own every upgrade and every on-call page |
| Adopt | Open source | Free to license, not free to own: integration, upgrades, maintenance |
| Buy | A vendor’s product or API | You pay for someone else’s scale and roadmap |
| Partner | A vendor builds with you, on your data | Shared control |
Our own stack, by source:
| Source | Components |
|---|---|
| Built | Gatekeeper and business rules · loop governor · guardrail orchestration · grounding checks · context pipeline · memory governance · analytics contract · eval oracle + golden set |
| Adopted | LangGraph · MCP SDK · DuckDB · Presidio · NLI and intent models · DeepEval · Promptfoo · Rill · Pydantic · sqlglot |
| Bought | Hosted foundation models · hosted safety classifiers · a tracing UI (optional) |
One data point the industry argues about: MIT NANDA’s The GenAI Divide: State of AI in Business 2025.
- ~95% of companies’ generative-AI pilots in its dataset showed little or no measurable P&L impact.
- Purchases from specialised vendors and partnerships succeeded about 67% of the time; internal builds about a third as often.
- More than half of budgets went to sales and marketing tools, while the biggest ROI was in back-office automation.
It is survey- and interview-based, and its definition of “success” has been debated. But the direction matches practice.
The three numbers a CFO will ask for, over a stated horizon (e.g. 3 years):
\[\text{ROI} = \frac{\text{benefit} - \text{cost}}{\text{cost}} \qquad \text{Payback} = \frac{\text{one-off cost}}{\text{monthly net benefit}} \qquad \text{NPV} = \sum_t \frac{\text{benefit}_t - \text{cost}_t}{(1 + r)^t}\]- ROI: how much you get back per dollar spent.
- Payback: months until it has paid for itself.
- NPV: money later is worth less than money now; discounting at rate \(r\) says by how much.
Where an agent’s benefit comes from:
- Labour: deflected tickets, shorter handle time.
- ⏩ Cycle time: faster refunds, faster research.
- Revenue: conversion, retention.
- Risk avoided: losses prevented.
TCO is everything something costs over its life, not the sticker price:
| Build | Buy | |
|---|---|---|
| Up front | Engineers × months | Integration · onboarding |
| Running | Infrastructure · on-call · upgrades | Seats or usage, growing with volume |
| Hidden | Evals, red-teaming, model regressions | Workarounds for gaps · data egress · price rises |
| Leaving | Nothing to leave | Exit cost: data, rewrites |
| Always | Opportunity cost: what those engineers didn’t build | Vendor risk |
Costs specific to agents, each an instance of an earlier chapter:
| Cost | Why it recurs |
|---|---|
| Model churn | Every model change can regress behaviour, so you need evals to change models safely |
| Guardrail upkeep | Attack phrasing evolves (Agent Security) |
| Policy drift | Rules and golden labels must follow policy changes |
| Trace volume | Per-trace fees grow with traffic (Observability) |
| Human review | Escalations need people (Levels of Automation) |
The one formula that settles most cost arguments. Build has a high fixed cost (a team) and a low variable cost (tokens); buy has a low fixed cost and a high variable cost (a per-resolution fee):
\[C_{\text{build}} = F_b + v_b Q \qquad C_{\text{buy}} = F_v + v_v Q \qquad\Rightarrow\qquad Q^* = \frac{F_b - F_v}{v_v - v_b}\]Below \(Q^*\), buy is cheaper; above it, build is.
Plan for the price curve. The cost of a given level of model capability has been falling roughly 10× per year (a16z, LLMflation, 2024):
- Don’t build what the curve will commoditise, e.g. a custom-trained or fine-tuned model to save tokens: it competes with a price that falls 10× a year.
- Do build what the curve doesn’t touch: rules, data, evals, authority. Their value rises as tokens get cheaper, because cheaper tokens mean more agent traffic to govern.
- Keep switching cheap: models and judges behind configuration, so you can ride the curve.
| Factor | The question to ask |
|---|---|
| Differentiation | Does it make us better than competitors, or is it plumbing? |
| Liability · control | If it’s wrong, who pays, and can we explain the decision to a regulator? |
| Data sensitivity | May this data leave our boundary at all? |
| Time-to-value | What does a six-month build cost in delayed benefit? |
| Talent | Can we hire and keep the people to run it? |
| Volume | Where are we relative to \(Q^*\), now and later? |
| Rate of change | Is the field moving faster than we can keep up? |
| Exit cost | If the vendor triples its price, what does leaving cost? |
Differentiation × maturity sorts most components:
| Low differentiation for us | High differentiation for us | |
|---|---|---|
| Mature market | Buy / adopt (a database, a BI tool, a safety classifier) | Buy the core, build the edge (a bought model; your prompts, skills, evals around it) |
| Immature market | Adopt open source, stay thin, and wait for the market | Build (your authority rules, your grounding rules) |
1 · The authority layer (Gatekeeper, business rules, mandates, limits). The liability is yours, and no vendor knows your refund policy, limits or escalation paths. The economics:
\[\text{expected loss} = P(\text{wrong action}) \times \text{cost per wrong action} \times \text{volume}\]2 · Your evaluation data. Adopt the frameworks (DeepEval, Promptfoo); build the data: golden cases labelled from your policy, oracle rules, attacks against your tools. Nobody can sell you that. A core eval run costs minutes and a few dollars; one bad model upgrade that refunds the wrong order for a day can cost more than a year of runs.
3 · Domain checks.
| Check | Generic API | Ours |
|---|---|---|
| Refund-state claims | Doesn’t know the state machine | Direct record check · ~0 per call |
| Prose grounding | A hosted call per reply | Local NLI · ~0 per call |
| Jailbreak · safety | A maintained classifier | Bought |
4 · Facts and adapters. The analytics contract is cheap to build, near-free to run and vendor-neutral (sample the vendor’s trace view; keep the full record yourself). Thin adapters around every bought service (model chain, judge client, guard chain) make switching a configuration change, not a rewrite.
Models and inference: buy capability. Training a frontier model is orders of magnitude beyond an agent team’s budget, and prices keep falling. Self-hosting open weights has its own break-even:
\[\text{self-host pays when}\quad \frac{\text{GPU \$/hour}}{\text{tokens served/hour}} < \text{API \$/token}\quad \text{at real, sustained utilisation}\]An idle GPU still bills. Self-host only for utilisation, data residency, or a missing feature (e.g. constrained decoding). Small local models (intent classifiers, NLI, summarisers) are the exception worth running yourself: tiny, CPU-only, zero marginal cost, called every turn.
Adopt open-source plumbing: a graph runtime with checkpoints, a protocol SDK, an analytics engine, NER, validation, a SQL parser, eval runners. Zero licence cost; you pay in integration and upgrades. Wrap it where it touches the domain (a durability runner around the checkpointer, a governed source around the MCP SDK), so an upgrade or swap stays local.
Buy specialised classifiers and trace UIs: a safety classifier’s value is an adversarial dataset that keeps up with new attacks; building that means labelling and retraining forever. Keep deterministic rules underneath and fail closed. A trace UI pays when debugging hours are real; sample it.
When buying pays:
| Trigger | Because |
|---|---|
| \(Q < Q^*\) | At low volume, their variable cost beats your fixed cost |
| Time-to-value | Live in weeks vs quarters to build |
| Non-differentiating | Customers never see plumbing |
| Their scale | Shared R&D: datasets, training |
| Certifications | SOC 2, ISO 27001, HIPAA: costly to earn alone |
| Talent gap | You can’t staff it well |
Whatever you buy, negotiate the exit: data export, price caps, an adapter on your side.
Read more about Supervised Fine-Tuning (SFT)
Read more about Parameter-Efficient Fine-Tuning (PEFT)
Read more about Large Language Models
Every number is illustrative; the method is the point. Plug in your own.
Inputs: 50,000 tickets/month; the agent fully resolves 40% → 20,000 resolutions; each saves $5 of human handling → $100,000/month benefit.
| Build | Buy | |
|---|---|---|
| One-off | $360k (4 engineers × 6 months × $15k) | $60k (2 engineers × 2 months) |
| Team · infra | $48k/month (3 engineers + infra) | $15k/month (1 engineer) |
| Per resolution | $0.125 (tokens, guards) | $1.00 (vendor fee) |
| Monthly cost @ 50k tickets | 48k + 0.125 × 20k = $50.5k | 15k + 1.00 × 20k = $35k |
| Payback | 360 ÷ (100 − 50.5) ≈ 7.3 months | 60 ÷ (100 − 35) ≈ < 1 month |
| 3-year ROI | (3.6M − 2.18M) ÷ 2.18M ≈ 65% | (3.6M − 1.32M) ÷ 1.32M ≈ 173% |
At this volume, buy wins on money. Now change one input, volume:
| Tickets/month | Build | Buy | Winner |
|---|---|---|---|
| 50k | $50.5k | $35k | Buy |
| 100k | $53k | $55k | Roughly even |
| 200k | $58k | $95k | Build, by $37k every month |
The crossover is the break-even from earlier, about 94k tickets a month.
This course built most things because it is a teaching platform: you can’t explain a Gatekeeper you didn’t write. That’s a legitimate reason here, and not one a product team has. For a production team:
| Build | The control plane and the data: authority, domain checks, the analytics contract, golden sets, attacks, policies |
| Adopt | Open-source plumbing, wrapped at the domain edge |
| Buy | Models, safety classifiers, trace UIs, behind adapters, with the exit negotiated |
| Revisit | At every order-of-magnitude change in volume or price |
- Where are they relative to \(Q^*\)? At 30k tickets the fixed cost of a team and GPUs may never pay back.
- The price curve: a saving that competes with prices falling ~10× a year evaporates fast.
- Utilisation: will self-hosted GPUs run at sustained high utilisation, or bill while idle?
- Is it differentiating? Probably not: the model is commodity. The money is better spent on what the curve doesn’t touch (authority rules, domain checks, eval data) and on an adapter that keeps switching to a cheaper hosted model a config change.
- Decide per component: build, adopt, buy or partner.
- Know the arithmetic: ROI, payback, NPV, TCO, and break-even \(Q^*\).
- Build the control plane and the data: their value rises as tokens get cheaper.
- Buy the commodity, behind adapters, with the exit negotiated.
You can rent intelligence. You can’t rent accountability.
- MIT NANDA, The GenAI Divide: State of AI in Business 2025 (2025), preliminary and survey-based.
- Fortune, MIT report: 95% of generative AI pilots at companies are failing (18 Aug 2025).
- Appenzeller (a16z), Welcome to LLMflation: LLM inference cost is going down fast (Nov 2024).
You've worked through all 16 lessons in this module.