Fine-Tuning ≠ Knowledge Injection (I Built a Demo to Prove It)

There’s a sentence I keep hearing in AI conversations, and it’s almost always said with complete confidence:

“We’ll just fine-tune the model on our data, and then it’ll know our business.”

It sounds right. It feels right. Train the model on your Salesforce data, and now it knows your accounts, right?

It doesn’t work. And instead of arguing about it in meetings, I decided to build the argument as a thing you can click on — a live demo, running on the RTX 3060 in my home server, where you can watch a real fine-tuned model fail to know things it was literally trained on.

The demo is live here — served from the same GPU that trained it.

The experiment design

The tricky part of proving “the model doesn’t know your data” is proving the model couldn’t have known the data some other way. If I fine-tuned on real company data, a skeptic could always say the base model had seen something like it during pretraining.

So the whole world is synthetic. Solstice Software — a fake B2B SaaS company with 30 accounts, ~150 support cases, 20 policy articles, and 212 graded questions, all generated deterministically by a seeded script. The base model provably cannot know any of it, because it didn’t exist until the script ran.

Then the split that makes the experiment interesting:

• 8 accounts are held-in — their facts appear in the fine-tuning data. This is the punchline account: even records the model was trained on don’t stick.
• 22 accounts are held-out — including the hero account, Brightloom Retail Group, which appears nowhere in training.

I fine-tuned Qwen3-4B with QLoRA (via Unsloth) to become “Astra” — a CRM copilot with a very recognizable voice: a bold snapshot line, structured bullets, and a “Next best action:” closer on every answer. The persona is deliberately low-entropy — I wanted the style transfer to be undeniable in an A/B comparison.

Training takes about 4 minutes on the 3060. That part of the pitch is true — fine-tuning really is cheap now.

It took four adapters to get one that worked

The first adapter came out speaking fluent Astra… and opening every answer with degenerate <tool_call> tag spam.

This kicked off the most satisfying debugging arc of the project:

• v1: Blamed the tool-calling examples in the training data (26% of the mix). Too much dose, overfit.
• v2: Cut the tool examples down, doubled the voice examples. Better — but answers still opened with one or two stray tags.
• v3: Dropped the tool-format slice entirely. Zero <tool_call> strings anywhere in the training data. The tags still showed up.

That’s the moment you know your theory is wrong. If the corruption survives removing its supposed cause, the cause is somewhere else.

The real culprit: a chat template mismatch. Unsloth’s quantized checkpoint shipped a tokenizer that renders assistant turns with a <think> block — a thinking template on a non-thinking model. vLLM serves the original Qwen template, without it. All three adapters had been trained to emit a think-prefix that never exists at serving time, and the garbage tags were its mangled ghost. One line — overwrite the chat template with the original before rendering — and v4 came out clean. Perfect voice, zero stray tokens.

(There was also a detour where vLLM segfaulted because the default PyPI wheels wanted a newer NVIDIA driver than my box has. The fix — a cu129 release wheel plus torch from the cu128 index — is documented in the repo for the next person. Claude Code did the heavy lifting through this whole saga; my job was mostly reading logs with it, going “huh”, and testing the next theory.)

The results

Everything below is auto-graded and generated live on the 3060 — the demo re-runs it on demand.

Read that middle column carefully, because it’s the whole story:

1. The voice transferred perfectly. 100% of answers carry the Astra signature. Fine-tuning absolutely works — for behavior.
2. Knowledge didn’t move at all. 0/22 on held-out records. And the row that should end the argument: 0/20 on records that were in the training data. Even the facts the model saw during fine-tuning didn’t stick.
3. It got worse than the base model in the way that matters. The base model refused half the record questions — “I don’t have access to that.” The fine-tuned model never refuses. It answers everything, in a polished, confident, structured format, and it is wrong every time.

Ask Astra when Brightloom Retail Group renews and you get a beautiful snapshot line, a renewal date, a usage pattern, and a recommended next action. The date is invented. The truth is 2027-03-12; I’ve watched it confidently say September 2024. The fine-tune taught the model to sound like it knows — which is strictly more dangerous than not knowing.

What actually injects knowledge

The same fine-tuned model, same GPU, same adapter — with two changes that don’t touch the weights at all:

• Tool-calling (agentic): give Astra get_account, search_cases, search_kb over the CRM store. 14/14 record questions correct. The persona survives — the snapshot line is now wrapped around retrieved facts.
• RAG: embed the 20 policy articles (bge-small, LanceDB), retrieve at answer time. 10/11 policy questions correct.

Fine-tuning owns behavior. Retrieval owns knowledge. They’re not competing techniques — they’re different tools for different layers of the problem, and the expensive mistake is using the first one for the second one’s job.

If you want the decision-maker version of this argument — when fine-tuning does pay, what LoRA/QLoRA actually are, why “free open-weight model” ≠ free to run — I put together a plain-English field guide deck that covers the mental models without any code.

Try it

varr.run/projects/fine-tuning-vs-rag — six acts: the data, the training run (real loss curve), behavior transfer ✅, knowledge failure ❌, grounding ✅✅, and a scoreboard that re-grades everything live.

It runs on a single RTX 3060 with 12GB of VRAM, self-hosted from my home server through a Cloudflare tunnel — vLLM serving the base model and the LoRA adapter side by side, FastAPI orchestrating, the same machine that trained the adapter in the first place. If the demo says the GPU is offline, the box is probably busy doing something else — check back.

Source is on GitHub, including the build log with every failure documented, because the failures were the educational part.