Working without the internet: 4 local AIs that work in a chain

Why plan for a fallback mode

I use an online AI for my personal projects in the evenings and at weekends: coding, writing, doing SEO on my websites. The day my internet box gives up, everything stops. An internet outage should no longer block my projects.

I had already installed a local AI for coding (Qwen 2.5 Coder 3B with Ollama). This time I went further: a small panel of 4 models, each with its own role, working one after the other on the same task.

The result doesn’t match an online AI. But it lets me keep moving in fallback mode, for free, and without any data leaving my PC.

My setup: 4 GB of VRAM, the real limit

Component Model
Processor Intel Core i5-11400F (6 cores, 12 threads)
RAM 32 GB
Graphics card NVIDIA GTX 1650 Super, 4 GB of VRAM
System Windows, with Ollama and VS Code

The graphics card’s memory decides everything. A model that fits entirely in the 4 GB answers fast. If it overflows, part of it moves to the processor and speed drops.

Two rules follow from this:

  • models with 2 to 4 billion parameters for everyday work;
  • only one model loaded at a time, so the graphics card is never saturated.

The 4 models and the role of each

Each model has a strength, and the chain combines them.

Model Ollama command Role in the chain Why this one
Qwen 3.5 4B qwen3.5:4b Plan, then finalize The best reasoning at this size, with a “thinking” mode you can switch on
Gemma 4 E2B gemma4:e2b Write Lightweight, good at writing, and it can also read images
Phi-4 Mini phi4-mini Proofread and critique Trained on logic: a different perspective from the Qwen models
Qwen 3.5 9B qwen3.5:9b Backup for difficult tasks Smarter, but overflows the card: slower

I kept Qwen 2.5 Coder 3B for autocompletion in VS Code, and added nomic-embed-text, a small model used to search through a project’s code.

Install the models with one command

With Ollama already installed, a single PowerShell line downloads the whole panel:

"qwen3.5:4b", "gemma4:e2b", "phi4-mini", "qwen3.5:9b", "nomic-embed-text" | ForEach-Object { ollama pull $_ }

Plan for around 15 GB of disk space. If the download is interrupted, just run the command again: models already present aren’t downloaded twice.

To check that everything is there:

ollama list

Do this before the outage: this download needs the internet. Fallback mode is prepared while everything works.

Direct answers, no outline

In thinking mode, Qwen 3.5 writes a long chain of reasoning before answering. For everyday use, I created a variant that answers directly:

@"
FROM qwen3.5:4b
SYSTEM "Answer in English, directly and briefly. No outline, no introduction."
"@ | Out-File -Encoding utf8 Modelfile

ollama create qwen-fast -f Modelfile
ollama run qwen-fast --think=false

Making the AIs work one after the other

A small Python script passes the task from one model to the next, and unloads each model from the graphics card as soon as it’s done. So there are never two in memory at the same time.

The chain follows four steps:

  1. Qwen 3.5 4B breaks the request down into a short plan.
  2. Gemma 4 E2B writes following that plan.
  3. Phi-4 Mini proofreads and lists errors and omissions.
  4. Qwen 3.5 4B fixes the text based on the critique and returns the final version.

Each model receives the work of the previous ones. The full result is saved in a result.md file.

The script

It only uses standard Python: nothing extra to install. Two parameters do all the work: keep_alive: 0 frees the graphics card after each answer, and think: False avoids long reasoning.

import json, sys, urllib.request

OLLAMA_URL = "http://localhost:11434/api/chat"

STEPS = [
    ("Planner", "qwen3.5:4b",
     "You are a planner. Break the request down into a short, concrete plan (5 points max)."),
    ("Writer", "gemma4:e2b",
     "You are a writer. Produce the requested work following the plan provided."),
    ("Critic", "phi4-mini",
     "You are a demanding proofreader. List the errors, omissions and possible improvements. Be brief."),
    ("Finalizer", "qwen3.5:4b",
     "You are the final editor. Fix the work taking the critique into account. "
     "Return only the final version."),
]

def call(model, system, message):
    body = json.dumps({
        "model": model,
        "messages": [
            {"role": "system", "content": system + " Answer in English."},
            {"role": "user", "content": message},
        ],
        "stream": False,
        "think": False,
        "keep_alive": 0,
    }).encode("utf-8")
    req = urllib.request.Request(OLLAMA_URL, data=body,
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=900) as resp:
        return json.loads(resp.read())["message"]["content"].strip()

request = sys.argv[1]
context = f"REQUEST:\n{request}"
results = {}
for name, model, role in STEPS:
    print(f"\n=== {name} ({model}) running... ===")
    answer = call(model, role, context)
    results[name] = answer
    print(answer)
    context += f"\n\n--- {name.upper()} ---\n{answer}"

with open("result.md", "w", encoding="utf-8") as f:
    f.write(f"# Request\n{request}\n\n")
    for name, text in results.items():
        f.write(f"## {name}\n{text}\n\n")

Running it

py agents.py "Write a 600-word article about renting a photo booth for a wedding"

To change the chain, just edit the STEPS list: add a step, change a role, or hand the final step to qwen3.5:9b for better quality.

The same models in VS Code

In VS Code, the Continue extension uses Ollama’s models without a connection. I split the roles there: the small code model for autocompletion, Qwen 3.5 for chat.

Continue’s config.yaml file (in %USERPROFILE%\.continue\):

name: Local AI
version: 1.0.0
schema: v1
models:
  - name: Qwen Coder 3B (local)
    provider: ollama
    model: qwen2.5-coder:3b
    roles:
      - autocomplete
      - apply
  - name: Qwen Fast
    provider: ollama
    model: qwen-fast
    roles:
      - chat
      - edit
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text
    roles:
      - embed

With nomic-embed-text, Continue can also search the whole project with @codebase.

Results, and the limits

Fallback mode delivers: I can keep coding, writing and structuring content for my personal projects during an outage. But you have to accept a level below an online AI.

What works well

  • Understanding and debugging a piece of code, an .htaccess rule, an SQL query.
  • Producing a first draft of an article, title and meta tags, a heading structure.
  • Rewording a text or an email.
  • Working on confidential content: nothing leaves the PC.

The limits

  • Speed: the chain runs four models and reloads them at each step. On my machine, the full chain took 6 min 55 s to produce an article of about 1,000 words.
  • Thinking mode: when on, it makes Qwen much slower. I keep it for difficult questions.
  • Knowledge: these small models easily make up facts and have no recent information. Anything factual gets checked once the connection is back.
  • Context: with 4 GB of VRAM, you can’t give them long documents.

My tips

  • Install and test everything before you need it.
  • Check with ollama ps that the model runs 100% on the graphics card.
  • Close games and the browser while the chain runs, to free up VRAM.

The test: I had my local AIs write this article

The chain held up technically, but the text it produced wasn’t publishable. I gave the 4 models a single topic: write a 1,000-word article about this setup. They took 6 min 55 s, one after the other, without saturating the graphics card.

The roles weren’t respected

  • The planner wrote the whole article instead of a plan.
  • The critic rewrote everything instead of listing the problems.
  • The finalizer commented on the text instead of returning the final version.

Made-up technical details

  • An “LLM-STD protocol” that doesn’t exist.
  • A --numba option that doesn’t exist either.
  • “Q4 is impossible on 4 GB”: wrong, my Qwen 4B runs in Q4.
  • “Ollama chains the steps on its own”: wrong, my script does that.
  • A “week-long outage” I never went through.

Outdated knowledge: the finalizer, itself a Qwen 3.5, claimed that Gemma 4 and Qwen 3.5 “don’t exist yet”. Its knowledge stops before they were released.

Why

I had given them a topic, not the facts. A small model that doesn’t know fills the gaps by making things up. Loose role instructions and an ever-growing history did the rest.

What I take away

I bring the facts, the local AI shapes them.

My local AI is reliable for It is not reliable for
Rewording or shortening a text I wrote Writing an article from a bare topic
Turning my notes into paragraphs and headings Talking about recent tech
Suggesting titles, meta descriptions and headings Following complex roles in a chain
Checking spelling and phrasing Giving facts without verification
Understanding and debugging code

FAQ

Do you need a big graphics card to run several local AIs?

No. By making them work one after the other, a 4 GB card is enough, because only one model is loaded at a time.

Do local AIs really work without the internet?

Yes. Once the models are downloaded, Ollama, Continue and the script run entirely offline.

Why use several models instead of just one?

Each small model has its strengths. Having them check each other’s work catches some of the errors a single model would let through.

Does it replace an online AI?

No. It’s a fallback mode: useful to keep moving, but slower and less reliable.

How much disk space do you need?

About 15 GB for the five models covered here.

Conclusion

With four small models, Ollama and a script of about fifty lines, an internet outage no longer blocks my personal projects. The result stays below an online AI, but it’s free, private and always available. Two lessons: prepare everything before the outage, and stay the author. During an outage, the local AI is a writing assistant, not a writer.