Why plan for a fallback mode
I use an online AI for my personal projects in the evenings and at weekends: coding, writing, doing SEO on my websites. The day my internet box gives up, everything stops. An internet outage should no longer block my projects.
I had already installed a local AI for coding (Qwen 2.5 Coder 3B with Ollama). This time I went further: a small panel of 4 models, each with its own role, working one after the other on the same task.
The result doesn’t match an online AI. But it lets me keep moving in fallback mode, for free, and without any data leaving my PC.
My setup: 4 GB of VRAM, the real limit
| Component | Model |
|---|---|
| Processor | Intel Core i5-11400F (6 cores, 12 threads) |
| RAM | 32 GB |
| Graphics card | NVIDIA GTX 1650 Super, 4 GB of VRAM |
| System | Windows, with Ollama and VS Code |
The graphics card’s memory decides everything. A model that fits entirely in the 4 GB answers fast. If it overflows, part of it moves to the processor and speed drops.
Two rules follow from this:
- models with 2 to 4 billion parameters for everyday work;
- only one model loaded at a time, so the graphics card is never saturated.
The 4 models and the role of each
Each model has a strength, and the chain combines them.
| Model | Ollama command | Role in the chain | Why this one |
|---|---|---|---|
| Qwen 3.5 4B | qwen3.5:4b |
Plan, then finalize | The best reasoning at this size, with a “thinking” mode you can switch on |
| Gemma 4 E2B | gemma4:e2b |
Write | Lightweight, good at writing, and it can also read images |
| Phi-4 Mini | phi4-mini |
Proofread and critique | Trained on logic: a different perspective from the Qwen models |
| Qwen 3.5 9B | qwen3.5:9b |
Backup for difficult tasks | Smarter, but overflows the card: slower |
I kept Qwen 2.5 Coder 3B for autocompletion in VS Code, and added nomic-embed-text, a small model used to search through a project’s code.
Install the models with one command
With Ollama already installed, a single PowerShell line downloads the whole panel:
"qwen3.5:4b", "gemma4:e2b", "phi4-mini", "qwen3.5:9b", "nomic-embed-text" | ForEach-Object { ollama pull $_ }
Plan for around 15 GB of disk space. If the download is interrupted, just run the command again: models already present aren’t downloaded twice.
To check that everything is there:
ollama list
Do this before the outage: this download needs the internet. Fallback mode is prepared while everything works.
Direct answers, no outline
In thinking mode, Qwen 3.5 writes a long chain of reasoning before answering. For everyday use, I created a variant that answers directly:
@"
FROM qwen3.5:4b
SYSTEM "Answer in English, directly and briefly. No outline, no introduction."
"@ | Out-File -Encoding utf8 Modelfile
ollama create qwen-fast -f Modelfile
ollama run qwen-fast --think=false
Making the AIs work one after the other
A small Python script passes the task from one model to the next, and unloads each model from the graphics card as soon as it’s done. So there are never two in memory at the same time.
The chain follows four steps:
- Qwen 3.5 4B breaks the request down into a short plan.
- Gemma 4 E2B writes following that plan.
- Phi-4 Mini proofreads and lists errors and omissions.
- Qwen 3.5 4B fixes the text based on the critique and returns the final version.
Each model receives the work of the previous ones. The full result is saved in a result.md file.
The script
It only uses standard Python: nothing extra to install. Two parameters do all the work: keep_alive: 0 frees the graphics card after each answer, and think: False avoids long reasoning.
import json, sys, urllib.request
OLLAMA_URL = "http://localhost:11434/api/chat"
STEPS = [
("Planner", "qwen3.5:4b",
"You are a planner. Break the request down into a short, concrete plan (5 points max)."),
("Writer", "gemma4:e2b",
"You are a writer. Produce the requested work following the plan provided."),
("Critic", "phi4-mini",
"You are a demanding proofreader. List the errors, omissions and possible improvements. Be brief."),
("Finalizer", "qwen3.5:4b",
"You are the final editor. Fix the work taking the critique into account. "
"Return only the final version."),
]
def call(model, system, message):
body = json.dumps({
"model": model,
"messages": [
{"role": "system", "content": system + " Answer in English."},
{"role": "user", "content": message},
],
"stream": False,
"think": False,
"keep_alive": 0,
}).encode("utf-8")
req = urllib.request.Request(OLLAMA_URL, data=body,
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=900) as resp:
return json.loads(resp.read())["message"]["content"].strip()
request = sys.argv[1]
context = f"REQUEST:\n{request}"
results = {}
for name, model, role in STEPS:
print(f"\n=== {name} ({model}) running... ===")
answer = call(model, role, context)
results[name] = answer
print(answer)
context += f"\n\n--- {name.upper()} ---\n{answer}"
with open("result.md", "w", encoding="utf-8") as f:
f.write(f"# Request\n{request}\n\n")
for name, text in results.items():
f.write(f"## {name}\n{text}\n\n")
Running it
py agents.py "Write a 600-word article about renting a photo booth for a wedding"
To change the chain, just edit the STEPS list: add a step, change a role, or hand the final step to qwen3.5:9b for better quality.
The same models in VS Code
In VS Code, the Continue extension uses Ollama’s models without a connection. I split the roles there: the small code model for autocompletion, Qwen 3.5 for chat.
Continue’s config.yaml file (in %USERPROFILE%\.continue\):
name: Local AI
version: 1.0.0
schema: v1
models:
- name: Qwen Coder 3B (local)
provider: ollama
model: qwen2.5-coder:3b
roles:
- autocomplete
- apply
- name: Qwen Fast
provider: ollama
model: qwen-fast
roles:
- chat
- edit
- name: Nomic Embed
provider: ollama
model: nomic-embed-text
roles:
- embed
With nomic-embed-text, Continue can also search the whole project with @codebase.
Results, and the limits
Fallback mode delivers: I can keep coding, writing and structuring content for my personal projects during an outage. But you have to accept a level below an online AI.
What works well
- Understanding and debugging a piece of code, an .htaccess rule, an SQL query.
- Producing a first draft of an article, title and meta tags, a heading structure.
- Rewording a text or an email.
- Working on confidential content: nothing leaves the PC.
The limits
- Speed: the chain runs four models and reloads them at each step. On my machine, the full chain took 6 min 55 s to produce an article of about 1,000 words.
- Thinking mode: when on, it makes Qwen much slower. I keep it for difficult questions.
- Knowledge: these small models easily make up facts and have no recent information. Anything factual gets checked once the connection is back.
- Context: with 4 GB of VRAM, you can’t give them long documents.
My tips
- Install and test everything before you need it.
- Check with
ollama psthat the model runs 100% on the graphics card. - Close games and the browser while the chain runs, to free up VRAM.
The test: I had my local AIs write this article
The chain held up technically, but the text it produced wasn’t publishable. I gave the 4 models a single topic: write a 1,000-word article about this setup. They took 6 min 55 s, one after the other, without saturating the graphics card.
The roles weren’t respected
- The planner wrote the whole article instead of a plan.
- The critic rewrote everything instead of listing the problems.
- The finalizer commented on the text instead of returning the final version.
Made-up technical details
- An “LLM-STD protocol” that doesn’t exist.
- A
--numbaoption that doesn’t exist either. - “Q4 is impossible on 4 GB”: wrong, my Qwen 4B runs in Q4.
- “Ollama chains the steps on its own”: wrong, my script does that.
- A “week-long outage” I never went through.
Outdated knowledge: the finalizer, itself a Qwen 3.5, claimed that Gemma 4 and Qwen 3.5 “don’t exist yet”. Its knowledge stops before they were released.
Why
I had given them a topic, not the facts. A small model that doesn’t know fills the gaps by making things up. Loose role instructions and an ever-growing history did the rest.
What I take away
I bring the facts, the local AI shapes them.
| My local AI is reliable for | It is not reliable for |
|---|---|
| Rewording or shortening a text I wrote | Writing an article from a bare topic |
| Turning my notes into paragraphs and headings | Talking about recent tech |
| Suggesting titles, meta descriptions and headings | Following complex roles in a chain |
| Checking spelling and phrasing | Giving facts without verification |
| Understanding and debugging code |
FAQ
Do you need a big graphics card to run several local AIs?
No. By making them work one after the other, a 4 GB card is enough, because only one model is loaded at a time.
Do local AIs really work without the internet?
Yes. Once the models are downloaded, Ollama, Continue and the script run entirely offline.
Why use several models instead of just one?
Each small model has its strengths. Having them check each other’s work catches some of the errors a single model would let through.
Does it replace an online AI?
No. It’s a fallback mode: useful to keep moving, but slower and less reliable.
How much disk space do you need?
About 15 GB for the five models covered here.
Conclusion
With four small models, Ollama and a script of about fifty lines, an internet outage no longer blocks my personal projects. The result stays below an online AI, but it’s free, private and always available. Two lessons: prepare everything before the outage, and stay the author. During an outage, the local AI is a writing assistant, not a writer.