Chapter 11
11Building Models: When to Build and When to Use Ready-Made
“Never start from scratch unless you have to.”
Every project faces a strategic decision: do we build a new “brain” from scratch, or use a ready expert one? This is “build vs. buy.” Building a deep-learning model from nothing consumes time, data and expensive compute, and fortunately we rarely need to today.
Text in this figure
Define · the problem · common problem · unique problem · Model garden + fine-tuning · a ready model specializes · in hours or minutes · Automated ML · tries hundreds of algorithms · and picks the best · ⚠ Avoid the cost of stupidity: don’t use a rocket to deliver pizza · The ladder of options in the foundation-model era · good prompt · RAG retrieval · LoRA fine-tune · distillation · train from scratch · simpler & cheaper · rarer & costlier · Figure 15
Option One: “Model Gardens” and Pre-training
The golden rule: do not start from scratch if your problem is “common.” “Model gardens” or “model hubs” are digital libraries of thousands of pre-trained models, trained on data beyond most organizations’ reach.
- When to use them: for general, recurring tasks: object recognition in images, sentiment analysis and translation.
- Choose: pick a general model, say one expert in millions of images.
- Fine-tune: light additional training on our data, say 1,000 photos of local bird species.
- The result: instead of months of training, the model specializes in our task within hours or minutes.
Option Two: “Automated Machine Learning”
What if your problem is unique and you have no team of data scientists? AutoML is an automated model maker, a middle path between ready-made and fully custom.
- When to use it: for unique data on a problem unlike any other, such as inventing a healthy cake recipe from given ingredients, or forecasting sales of an unprecedented product.
- We supply the data: a structured table of ingredients, flavors and recipe ratings.
- It builds: it tries hundreds of algorithms and combinations automatically to find the best fit.
- The result: a custom, high-performing model without writing a line of modelling code.
The Ladder of Options in the Foundation-Model Era
2026 Update
With generative models the decision has become a five-rung ladder, climbed rung by rung rather than jumped: (1) use a ready model with a good prompt, (2) give it your knowledge through RAG retrieval (Chapter 12), (3) fine-tune it efficiently on your data with light techniques such as LoRA, which adjust a small fraction of the weights, (4) “distill” a large model into a smaller, cheaper, faster one for a specific task, (5) pre-train from scratch, which is rare and costly and only for those with exceptional data and compute. On top of that comes the choice between a closed model via an API, or an open-weight model inside your environment when data sovereignty is a priority.
From the Field
The dark side of execution: before you press “run,” watch for three hidden traps. (1) The cost of stupidity: using a huge language model for simple arithmetic or a small database lookup is like using a space rocket to deliver pizza; if traditional software solves it, use it. (2) The legacy-systems battle: the biggest challenge is getting your agent to talk to the “old server” in the company basement; prepare for the integration battle before the intelligence battle. (3) The cloud-bill trap: costs look trivial at first, but when the service succeeds the bill can devour your profits. Calculate unit economics from day one.
Lessons Learned
- 1Never start from scratch unless you have to.
- 2For common problems: a pre-trained model from a model garden, plus fine-tuning.
- 3For unique problems with structured data: automated machine learning.
- 4Climb the ladder gradually: prompt, then retrieval, then fine-tuning, then distillation, then building.
- 5Calculate cost, unit economics and integration difficulty before you start.
Tip: use ← → to move between sections.