Why a local AI for coding?
In less than an hour, I set up a coding assistant that runs entirely on my PC, with no subscription and no internet connection. It writes at 65 tokens per second on a 2019 graphics card with only 4 GB of memory.
I use online AI tools every day. But they have three drawbacks when you build things for clients:
- Privacy: your code goes to third-party servers. For a client project, that’s not a small thing.
- Cost: subscriptions and API credits add up fast.
- Dependency: no network, no assistant.
A local AI solves all three. The question was whether my rather modest PC would be enough. Here is the full step-by-step, with my real measurements.
My setup and what it allows
Graphics card memory (VRAM) decides everything. It sets the size of the model you can run fast.
| Component | My machine | Impact for AI |
|---|---|---|
| Graphics card | NVIDIA GTX 1650 Super, 4 GB VRAM | The limiting factor |
| RAM | 32 GB | Lets a bigger model overflow into it, at the cost of speed |
| Processor | Intel Core i5-11400F (6 cores, 12 threads) | Decent backup |
| System | Windows 64-bit | Natively supported by Ollama |
To check your own setup, open PowerShell and run:
Get-CimInstance Win32_Processor | Select-Object Name
[math]::Round((Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory/1GB,1)
nvidia-smi --query-gpu=name,memory.total --format=csv
With 4 GB of VRAM, you aim for models of around 3 billion parameters in Q4 quantization. I picked Qwen2.5-Coder 3B: it’s specialized in code, weighs 1.9 GB and fits entirely on the card.
Step 1: update the driver and install Ollama
A recent NVIDIA driver is essential for the AI to use the graphics card. Mine dated from 2022 (version 517). I updated it to version 617.14 from the NVIDIA website, then restarted.
Ollama is the software that downloads and runs models locally. On Windows, one command is enough:
winget install Ollama.Ollama
Close PowerShell, reopen it, then check the install:
ollama --version
nvidia-smi --query-gpu=driver_version --format=csv
If winget isn’t available, you can download the installer from ollama.com/download.
Step 2: first model and speed test
The model downloads and runs with the same command:
ollama run qwen2.5-coder:3b "Write a Python function that checks whether a string is a palindrome"
The download is 1.9 GB. The first load into memory takes a few seconds, then the answer appears word by word.
To check that the graphics card is doing the work, open a second PowerShell window:
ollama ps
On my machine, the model uses 2.2 GB and the PROCESSOR column shows 100% GPU. Nothing spills over to the CPU.
To measure speed, add the --verbose option:
ollama run qwen2.5-coder:3b --verbose "Write a JavaScript function that sorts an array of objects by date"
| Metric | Result |
|---|---|
| Generation speed (eval rate) | 65.86 tokens/s |
| Prompt reading (prompt eval rate) | 51.37 tokens/s |
| Tokens generated | 698 |
| Total duration | 10.9 s |
| Model load (already in memory) | 7 ms |
A complete function with explanations in under 11 seconds. That’s a comfortable reading speed, comparable to an online assistant.
Step 3: plug the AI into VS Code with Continue
The Continue extension connects VS Code to Ollama. It adds a code chat and autocomplete. I already had Cline, but it’s built for bigger models that work autonomously. With a 3B model, Continue is a better fit.
- In VS Code, open Extensions (
Ctrl+Shift+X), search for Continue and install the extension published by Continue. - Click the Continue icon in the sidebar. Don’t enter any API key: those are for online AIs.
- Open the config file via “Main Config”, or directly at
C:\Users\<you>\.continue\config.yaml. - Replace its content with:
name: Local AI
version: 1.0.0
schema: v1
models:
- name: Qwen Coder 3B (local)
provider: ollama
model: qwen2.5-coder:3b
roles:
- chat
- edit
- apply
- autocomplete
- Save, then pick Qwen Coder 3B (local) in the model selector.
I deliberately use the same model for chat and autocomplete. The card only loads one, which avoids saturating the 4 GB of VRAM.
Day-to-day use:
Ctrl+Lon selected code: ask a question in the chat.Ctrl+I: request an edit directly in the file.- As you type, a greyed-out suggestion appears.
Tabto accept it.
Use case: WordPress and SEO
This is where the local model helps me most day to day: short, repetitive technical tasks, often on client code I’d rather keep on my machine.
Setting it up for WordPress
Continue accepts permanent rules in config.yaml. I add this block under models: so every answer follows WordPress best practices:
rules:
- You are a senior WordPress developer. You answer in English.
- Follow the WordPress Coding Standards and prefix functions (e.g. pxv_).
- Always escape output (esc_html, esc_attr, esc_url) and sanitize input (sanitize_text_field).
- Use hooks (add_action, add_filter) rather than editing core or the parent theme.
Tip: in the chat, type @ followed by a file name (functions.php, a template) to give it to the model as context.
What works well for WordPress development
| Task | Example prompt |
|---|---|
| Hooks and snippets | “Add a filter that removes WordPress emojis from the head” |
| Custom post types | “Create a ‘project’ CPT with a ‘sector’ taxonomy” |
| Queries | “Write a WP_Query for the 3 latest posts in the SEO category” |
| Shortcodes | “Create a [cta] shortcode with a title and a link as parameters” |
| Debugging | Paste a PHP error from debug.log and ask for an explanation |
| Redirects | “Convert this list of URLs into 301 rules for .htaccess” |
What works well for technical SEO
| Task | Example prompt |
|---|---|
| Structured data | “Generate a LocalBusiness JSON-LD for a restaurant in Madrid” |
| Search Console regex | “Write a regex that filters queries containing quote or price” |
| Hreflang tags | “Generate the FR/EN/ES hreflang tags for this URL” |
| Processing exports | “Write a Python script that lists pages with no meta description in this CSV export” |
| Bulk tags | “Suggest a title under 60 characters for each of these 10 pages” |
What I don’t hand over to it
- Writing articles: Qwen2.5-Coder is specialized in code. Its prose is correct but flat, and it doesn’t know the latest SEO news.
- SEO strategy: keyword choice, competitor analysis, internal linking across a whole site. It lacks perspective and up-to-date data.
- Production code without review: I always check data escaping and test on a staging site before deploying.
In short: the local model is my assistant for WordPress code and technical SEO, not for content or strategy.
The limits, and my verdict
A 3B model is not a high-end online assistant. It’s fast and useful, but you need to know what to ask it.
A telling example: I asked for “a function that calculates 20% VAT on a pre-tax price”. It returned a function that calculates the VAT amount, not the price including tax. Correct, but minimal: a small model takes the request literally. You have to be precise in your instructions.
| Good for | Less good for |
|---|---|
| Autocomplete, short functions | Architecture of a whole project |
| Explaining a piece of code | Reasoning across several files |
| Regex, scripts, conversions | Autonomous agents (Cline and the like) |
| Working offline and privately | Vague or ambiguous requests |
To go further on the same machine, you can try qwen2.5-coder:7b. It’s more accurate, but it overflows from VRAM into RAM, so it will be slower.
My verdict: for a developer with a 4 GB graphics card, a local AI is already a real day-to-day gain. It complements an online assistant without replacing it: I keep local for client code and repetitive tasks, and the cloud for complex problems. All for free, in under an hour of setup.