Local AI in fallback mode: what I actually use it for every day

After my article about my 4 local AIs working in a chain, someone asked me a simple question: “On 4 GB of VRAM, I didn’t expect it to run 4 models in a chain. What do you use it for day to day in fallback mode?”

Good question. And the honest answer may disappoint a few people: I rarely run the full chain. Day to day, I mostly use one model at a time, for small, well-defined tasks. Here is what I actually do with them, and what I no longer ask them to do.

First, a note on the “4 models in a chain”

On 4 GB of VRAM, the models don’t run at the same time. They take turns: each one is loaded, does its job, then frees the card thanks to keep_alive: 0. That’s why it works, and also why it’s slow: almost 7 minutes for a 1,000-word text.

So I keep the full chain (plan, writing, review, final pass) for two cases:

  • Restructuring a long draft I’ve already written: I start it, go make a coffee, and read the result.
  • Preparing a FAQ page from my notes: Qwen makes the plan, Gemma writes, Phi-4 flags the vague answers.

For everything else, I call the right model directly. It’s faster, and most of my tasks don’t need three opinions.

One thing to know before going further: all these tasks may need checking. A small model makes mistakes, sometimes with great confidence. I always reread what it gives me before using it.

1. Coding without a connection

This is my number one use, by far. In VS Code, the Continue extension runs all the time with Qwen 2.5 Coder 3B for autocomplete. Without the internet, I barely notice the difference for everyday code.

What I actually ask it:

  • The little things I never remember: an .htaccess redirect rule, a regex to clean up URLs, an SQL query to list posts without a meta description.
  • Understanding my own code: with @codebase, I ask “where is the language switch handled?” in the WordPress theme of one of my personal projects. Handy when I come back to a project after a week away.
  • Debugging an error message: I paste the error and the file, Qwen 3.5 4B suggests a lead. It’s sometimes wrong, but it often saves me 10 minutes of searching.
  • My personal projects: a function for a WordPress theme, a bit of JavaScript, a small Python script to rename files.

Its limit: as soon as the question depends on a recent version of a library, it makes things up. For that, I wait for the connection to come back and read the official docs.

2. Writing: rewording, not writing for me

My last test proved it: asking for a whole article from a topic gives you a text full of made-up facts. Starting from my own notes, on the other hand, works well.

  • Messy notes → an outline: I type my ideas in no particular order, Qwen 3.5 4B suggests an H2/H3 structure. I keep what works for me.
  • Shortening a paragraph: “say the same thing in half the words.” Gemma 4 E2B is good at this.
  • Proofreading before publishing: Phi-4 Mini spots repetitions, sentences that are too long and vague passages. It doesn’t fix them, it flags them, which is exactly what I want.
  • A slightly tricky personal email: I write a first draft, then ask for a shorter, more neutral version.

The rule I set myself: local AI works on my text, never from scratch.

3. SEO for my articles

On-page SEO is repetitive, well-defined work: ideal ground for a small model.

  • Title and meta description: I paste the intro and ask for 5 options under 60 and 155 characters. I keep one and tweak it.
  • Checking the heading structure: I paste the article and ask whether any levels are skipped or any headings are too vague.
  • Image alt text: I describe the image in three words, it suggests a clean alt.
  • FAQ questions: “what questions would a reader ask after this article?” That’s often where I find the idea for the next one.

What it doesn’t do: keyword research. It has no search volume or competition data. It guesses, and I don’t want to build a strategy on a guess.

4. What never leaves my PC

This is the benefit I hadn’t thought of at first. Some texts I just don’t want to send to an online service:

  • my unpublished article drafts;
  • my personal notes and idea lists;
  • admin documents I just want to summarise or reword.

With Ollama, everything stays on the machine. Even with the internet on, I sometimes choose local for that.

What I never ask them to do

After a few weeks, the list is clear:

  • Facts or figures: dates, prices, versions, statistics. A 4-billion-parameter model makes things up with a straight face.
  • News: it knows nothing about what happened after its training.
  • An article from a bare topic: my test proved it, the result is flat and wrong in places.
  • A decision for me: it suggests, I decide.

In short: the more the task depends on what I give it, the better it works. The more it depends on what it “knows”, the less I trust it.

Recap: which task, which model

Task Model Approximate time
Autocomplete in VS Code Qwen 2.5 Coder 3B instant
.htaccess rule, regex, SQL Qwen 3.5 4B 10 to 20 s
Searching my code (@codebase) nomic-embed-text + Qwen 3.5 4B 20 to 40 s
Outline from my notes Qwen 3.5 4B under 1 min
Shortening, rewording Gemma 4 E2B 15 to 30 s
Critical review Phi-4 Mini under 1 min
Title, meta, alt, FAQ Qwen 3.5 4B under 1 min
Restructuring a long draft The full chain about 7 min

FAQ

Do you really need 4 models?

No. If I had to keep only one, it would be Qwen 3.5 4B, plus Qwen 2.5 Coder 3B for VS Code. The others are specialists I call on now and then.

Does it replace an online AI?

Not for me. Normally, I use an online AI for my evening and weekend projects. Local is the spare wheel, and the safe for whatever has to stay at home.

Is 4 GB of VRAM enough?

For one model of 2 to 4 billion parameters at a time, yes. Beyond that, it spills over into RAM and gets slow. 32 GB of RAM helps a lot.

Conclusion

Day to day, my local AI isn’t an article factory. It’s an assistant for small tasks: the forgotten bit of code, the meta description, the proofreading. I keep the 4-model chain for big drafts.

And when the connection drops, I keep moving forward instead of staring at the blinking router.