Do we need our own AI model? Why the answer is usually no
5 min read
Short answer
Most small and mid-sized companies do not need their own AI model. Ready-made models already understand language and documents very well. What they lack is your company knowledge, and that can be added more cheaply: through good instructions and examples, or through a search that passes the AI the relevant passages from your documents. Custom training rarely pays off.
What does this mean for your company? Book an intro call
“We need an AI that knows our company.” The wish is understandable, and it often leads to an expensive misunderstanding: a custom AI model. Yet the goal can almost always be reached more simply.
What options are there to connect AI with company knowledge?
There are four levels, with sharply rising effort:
- Instructions and examples: you tell the model how to work and give it a few good examples. This is surprisingly often enough.
- Knowledge search: before every answer, a system searches your documents and passes the relevant sections to the model. The AI answers on that basis and names the source.
- Fine-tuning: a ready-made model is adapted to a specific task or style using many examples.
- Custom training: a model is trained from scratch. That costs millions and is not an option for SMEs.
What is enough in most cases?
The combination of the first two levels. It has decisive advantages:
- Always up to date: a new document is available immediately, without retraining.
- Traceable: every answer can be traced back to a source.
- Swappable: when a better model appears, you simply switch. Your knowledge stays in your documents.
- Affordable: implementation is usually possible within a few weeks.
When does fine-tuning pay off?
It makes sense in a few cases: when a very specific task comes up in large volumes, such as sorting documents into hundreds of categories, when a fixed format must be followed reliably, or when a smaller, cheaper model is meant to take over a task. It requires several hundred to a few thousand checked examples.
What speaks against a custom model?
- Cost: training, infrastructure and specialists are out of all proportion to the benefit.
- Rapid ageing: new model generations appear at short intervals. A custom model is quickly outdated.
- Data protection: what has been trained into a model can hardly be removed selectively. That fits poorly with the right to erasure (Art. 17 GDPR) when personal data is involved.
And a model on our own servers?
That is a different question: it is about where processing happens, not about training. Freely available models can run on your own servers or in a German data centre. That can make sense for very sensitive data, but costs hardware or rent. The trade-offs are covered in Using AI in line with data protection, the costs in What does AI cost?
Julius Launhardt connects AI with the knowledge of small and mid-sized companies, without expensive detours via custom models. More on the AI consulting page.
Frequently asked questions
What is a knowledge search for AI?
Before every answer, the AI searches your documents, such as manuals or contracts, and receives the relevant sections. It answers on that basis and names the source. Specialists call this retrieval augmented generation, or RAG.
Does the AI learn from our documents?
Not with a knowledge search. The model stays unchanged and only reads your documents when needed. New documents are therefore available immediately, and deleted ones disappear from answers immediately.
How many examples does fine-tuning need?
Depending on the task, several hundred to a few thousand good, checked examples. Collecting them is often more work than the fine-tuning itself.
Is a model on our own servers the same as our own model?
No. Your own servers usually run a freely available, ready-made model. That concerns where processing happens, not training, and can make sense for very sensitive data.