Fine-Tuning a 0.6B Model to Run an Agent Entirely in Your Browser

It started, as usual, with something I saw on the internet: the bonsai-webgpu Space — a 1.7B-parameter LLM running entirely in the browser, a ~290 MB download, no server anywhere. You load a web page and there’s a language model living in your GPU.

Chatting with a tiny model in a browser tab is a neat trick. But I had a more interesting question:

Can a model this small do agentic work? Not chat — an actual loop: call tools, read the results, make a decision, cite its reasons?

The received wisdom says no — sub-1B models are toys. The research says something more nuanced: tiny models collapse in open-ended agent loops, but fine-tuned tiny models beat GPT-4-class models on narrow tool-calling tasks. TinyAgent did it. The xLAM family did it.

So the real question became: if I build the agent scaffolding right — a proper harness that owns the control flow, with the model just picking tools and making the final call — can I fine-tune a 0.6B model into a competent specialist that runs on your machine, in your browser, with zero server round-trips?

I picked an unreasonable task to find out: prior authorization review.

The demo is live here — your browser downloads the fine-tuned model (~1.5 GB, cached after the first load) and runs the whole agent loop on your GPU via WebGPU. My server just hands over static files. You’ll need a WebGPU-capable browser (Chrome or Edge, basically), and you can A/B the fine-tune against the stock model live. Why it’s a 1.5 GB download and not 600 MB is one of the better stories below.

Why prior auth, of all things

Prior auth is the process where a health insurer decides whether to approve a treatment before it happens. A reviewer takes a request, pulls the coverage criteria, checks the patient’s chart against them, and lands on approve / deny / needs-more-info — with cited criteria.

It’s a great agent task precisely because it’s not freeform: there’s a correct procedure (look up criteria → check codes → read the evidence → decide), a small set of tools, and a verifiable right answer. And it has a property I care about for in-browser AI: the data is sensitive. A reviewer agent that runs entirely on-device — where the chart never leaves the machine — is the kind of thing that only makes sense if local models can actually do the job.

To be clear about what this is: a research demo built on fully synthetic data — fake patients, real (public) Medicare coverage criteria, zero PHI. Nobody should adjudicate real claims with a 0.6B model. The point is to measure how far fine-tuning moves a tiny model on structured agentic work.

The architecture: a mini Claude-Code, in a browser tab

The design rule that makes tiny-model agents work: the harness owns the control flow; the model only owns the decisions.

Everything deterministic lives in JavaScript:

• A tool registry with five JSON-schema tools: lookup_criteria, get_patient, check_codes, and two terminals — request_info and decide.
• The loop: one tool call per model turn, parsed from Qwen’s native <tool_call> format, validated with AJV. Malformed call? The harness feeds the error back and retries, bounded.
• A context manager that maintains the running case file and truncates tool results.

The model runs in a Web Worker via Transformers.js with device: webgpu, greedy decoding. It sees the conversation, picks the next tool call (or the final decision), and that’s all it does.

If “a deterministic loop in JS driving an LLM that picks one tool at a time” sounds familiar — yes, it’s a tiny Claude Code, except the entire stack, model included, is a static web page.

The baseline: stock Qwen3-0.6B is genuinely terrible at this

Before fine-tuning anything, I ran stock Qwen3-0.6B through the harness with an ~800-token system prompt explaining the whole tool protocol.

It scored 12/40 (30%) on held-out cases — and even that flatters it. The failure modes were damning:

• It almost never denies. Deny recall: 1/16. It either rubber-stamps approvals or abstains — the one behavior that requires actually reading the record against the criteria is the one it can’t do.
• It often never looked at the patient. Entire episodes went criteria → codes → decision without a single get_patient call. Deciding without evidence, confidently.
• 27 malformed tool calls across 40 episodes, two of which died entirely.

That’s the “before” number. Everything from here is about moving it.

Building a dataset that doesn’t exist

There is no public dataset of prior-auth decisions. Nobody has published one — real PA data is PHI soup. But all the ingredients are public:

• Real coverage criteria from CMS’s Medicare Coverage Database — actual LCD policies for home oxygen, CPAP, CGM, and hospital beds, condensed into structured checklists with IDs.
• Synthetic patients, perturbed per-topic into four case shapes: clearly meets criteria, affirmatively fails one, borderline, and missing-documentation.
• Deterministic gold labels: a precedence rule — an affirmatively failed criterion ⇒ deny; a merely absent element ⇒ needs-more-info; otherwise approve. Executable code, not LLM vibes, decides the ground truth.

Then I generated ideal-reviewer trajectories through the real harness — the exact tool protocol, the exact JSON — and expanded them into 13,095 per-step training samples. About 8% are recovery samples: a malformed call and the harness’s error message sit in the context, and the correction is the training target. The model doesn’t just learn the happy path; it learns to fix its own mistakes.

Every completion was validated against the actual harness parser before training: 0 invalid samples, 0 train/eval case leakage.

Training on the 3060, or: the GPU crashed four times and it didn’t matter

LoRA on Qwen3-0.6B fits comfortably in 12 GB of VRAM. The fine-tune itself was almost boring — loss at 0.038 within 120 steps, token accuracy 98.9%. The protocol is extremely learnable when your dataset is per-step trajectories through the real harness.

The hardware was less boring. My RTX 3060 developed a habit of throwing CUDA “unspecified launch failure” after ~15–28 minutes of sustained training load. Twice in a row, mid-backward-pass.

Instead of chasing a driver fix at 6pm on a project day, I made training crash-tolerant rather than crash-free: checkpoint every 50 steps, auto-resume from the latest checkpoint, and a retry script that relaunches up to 12 times. The final run survived four GPU crashes and one session restart, resumed itself each time, and delivered the adapter after five attempts. Total wall-clock: about an hour and a half of babysitting a machine that was actively trying to die.

(Two bonus fights: newer transformers refuses to torch.load optimizer state on older torch for CVE reasons, and torch 2.5’s safe-unpickling allowlist doesn’t know about numpy RNG state. Both gated resuming from my own checkpoints. Both got surgically bypassed.)

Results: 30% → 77.5%, and zero malformed calls

Same harness, same 40 held-out cases, both models served at fp16 for a fair comparison:

The number I care most about is deny recall: 1/16 → 15/16. Denying correctly means the model read the record, found the failed criterion, and acted on it — the core competence of the task. And it does this from a 15-token system prompt; the entire tool protocol lives in the weights now. Across ~200 generations, not one malformed JSON call.

Where it still fails is honest and specific: 6 of the 9 errors are the model approving when a required element was merely absent from the record. Abstention-on-missing-evidence is the weakest behavior — and the obvious target for the next data iteration (oversample absent-element cases, add hard-negative pairs of the same case with and without the missing element).

Caveats, because they matter: n=40, cases come from the same generator as training (in-distribution), single greedy seed. The transfer test against real-world appeal adjudications is future work.

The boss fight I didn’t expect: quantization

Here’s where the weekend got interesting. The plan was always: fine-tune → merge → export to ONNX → quantize to 4-bit → ~600 MB browser download. The export path worked (after some version archaeology involving a vendored converter script from an old transformers.js tag).

Then the quantized model came back broken — call-looping, corrupted JSON, mid-episode protocol collapse. What followed was a full detective story of paired probes at every layer of the stack:

• Merged fp16 in Python: byte-perfect. Training was fine.
• ONNX q4, single-turn: perfect. Conversion was fine.
• ONNX q4, live multi-turn: the model starts bleeding tool-response content into its next tool call’s JSON. Context-dependent quantization damage — the longer the episode, the worse it gets.
• q8: coherent JSON, but the quantization noise un-trains the no-think behavior — the model flips back into <think> mode, which the fine-tune had specifically suppressed.

I found a 4-bit recipe (block size 16, asymmetric) that probed byte-identical to fp16 on every regression context I threw at it. Shipped it, ran the full eval: 6/40, 45 malformed calls. Clean for the first ~3 steps of each episode, then progressive collapse.

That’s the lesson I’ll carry to every future deployment: probe-level quantization checks lie. Only full-episode evals tell the truth. A quantized agent can be byte-perfect on short contexts and completely unusable at step 5.

So the browser build ships fp16 — a 1.5 GB download instead of 600 MB. Not the headline I wanted, but a true one. (Fixes worth exploring: quantization-aware LoRA — training against the quantized forward pass — or 8-bit weight-only matmuls.)

One last browser landmine for anyone following along: WebGPU requires a secure context. Testing from another machine over plain-HTTP LAN, navigator.gpu simply doesn’t exist. Dev server needed self-signed HTTPS before anything worked off-localhost.

What I actually learned

1. Tiny models + structured harnesses are real. A 0.6B model, fine-tuned on 13k trajectory steps, went from can’t-follow-the-protocol to 77.5% accuracy with zero protocol errors — running from a 15-token prompt.
2. The harness is half the model. Per-step training data generated through the real parser, with recovery samples, is why malformed calls went to zero. The protocol isn’t in the prompt anymore; it’s in the weights.
3. Fine-tuning encodes procedure, not facts. This is the same lesson as my last experiment from the other direction: there, fine-tuning failed to inject knowledge; here, it brilliantly injected behavior. Same mechanism, used for what it’s actually good at.
4. Quantize last, eval hardest. The cheapest-looking step of the pipeline ate two days and won.
5. Make training crash-tolerant, not crash-free. Checkpoints + auto-resume + a retry loop turned a dying GPU from a blocker into a nuisance.

All the code — harness, data generator, training scripts, evals — is on GitHub: varun1505/webgpu-browser-harness.