Running a local AI for coding on a modest PC (GTX 1650 Super)

Why a local AI for coding?

In less than an hour, I set up a coding assistant that runs entirely on my PC, with no subscription and no internet connection. It writes at 65 tokens per second on a 2019 graphics card with only 4 GB of memory.

I use online AI tools every day. But they have three drawbacks when you build things for clients:

  • Privacy: your code goes to third-party servers. For a client project, that’s not a small thing.
  • Cost: subscriptions and API credits add up fast.
  • Dependency: no network, no assistant.

A local AI solves all three. The question was whether my rather modest PC would be enough. Here is the full step-by-step, with my real measurements.

My setup and what it allows

Graphics card memory (VRAM) decides everything. It sets the size of the model you can run fast.

Component My machine Impact for AI
Graphics card NVIDIA GTX 1650 Super, 4 GB VRAM The limiting factor
RAM 32 GB Lets a bigger model overflow into it, at the cost of speed
Processor Intel Core i5-11400F (6 cores, 12 threads) Decent backup
System Windows 64-bit Natively supported by Ollama

To check your own setup, open PowerShell and run:

Get-CimInstance Win32_Processor | Select-Object Name
[math]::Round((Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory/1GB,1)
nvidia-smi --query-gpu=name,memory.total --format=csv

With 4 GB of VRAM, you aim for models of around 3 billion parameters in Q4 quantization. I picked Qwen2.5-Coder 3B: it’s specialized in code, weighs 1.9 GB and fits entirely on the card.

Step 1: update the driver and install Ollama

A recent NVIDIA driver is essential for the AI to use the graphics card. Mine dated from 2022 (version 517). I updated it to version 617.14 from the NVIDIA website, then restarted.

Ollama is the software that downloads and runs models locally. On Windows, one command is enough:

winget install Ollama.Ollama

Close PowerShell, reopen it, then check the install:

ollama --version
nvidia-smi --query-gpu=driver_version --format=csv

If winget isn’t available, you can download the installer from ollama.com/download.

Step 2: first model and speed test

The model downloads and runs with the same command:

ollama run qwen2.5-coder:3b "Write a Python function that checks whether a string is a palindrome"

The download is 1.9 GB. The first load into memory takes a few seconds, then the answer appears word by word.

To check that the graphics card is doing the work, open a second PowerShell window:

ollama ps

On my machine, the model uses 2.2 GB and the PROCESSOR column shows 100% GPU. Nothing spills over to the CPU.

To measure speed, add the --verbose option:

ollama run qwen2.5-coder:3b --verbose "Write a JavaScript function that sorts an array of objects by date"
Metric Result
Generation speed (eval rate) 65.86 tokens/s
Prompt reading (prompt eval rate) 51.37 tokens/s
Tokens generated 698
Total duration 10.9 s
Model load (already in memory) 7 ms

A complete function with explanations in under 11 seconds. That’s a comfortable reading speed, comparable to an online assistant.

Step 3: plug the AI into VS Code with Continue

The Continue extension connects VS Code to Ollama. It adds a code chat and autocomplete. I already had Cline, but it’s built for bigger models that work autonomously. With a 3B model, Continue is a better fit.

  1. In VS Code, open Extensions (Ctrl+Shift+X), search for Continue and install the extension published by Continue.
  2. Click the Continue icon in the sidebar. Don’t enter any API key: those are for online AIs.
  3. Open the config file via “Main Config”, or directly at C:\Users\<you>\.continue\config.yaml.
  4. Replace its content with:
name: Local AI
version: 1.0.0
schema: v1
models:
  - name: Qwen Coder 3B (local)
    provider: ollama
    model: qwen2.5-coder:3b
    roles:
      - chat
      - edit
      - apply
      - autocomplete
  1. Save, then pick Qwen Coder 3B (local) in the model selector.

I deliberately use the same model for chat and autocomplete. The card only loads one, which avoids saturating the 4 GB of VRAM.

Day-to-day use:

  • Ctrl+L on selected code: ask a question in the chat.
  • Ctrl+I: request an edit directly in the file.
  • As you type, a greyed-out suggestion appears. Tab to accept it.

Use case: WordPress and SEO

This is where the local model helps me most day to day: short, repetitive technical tasks, often on client code I’d rather keep on my machine.

Setting it up for WordPress

Continue accepts permanent rules in config.yaml. I add this block under models: so every answer follows WordPress best practices:

rules:
  - You are a senior WordPress developer. You answer in English.
  - Follow the WordPress Coding Standards and prefix functions (e.g. pxv_).
  - Always escape output (esc_html, esc_attr, esc_url) and sanitize input (sanitize_text_field).
  - Use hooks (add_action, add_filter) rather than editing core or the parent theme.

Tip: in the chat, type @ followed by a file name (functions.php, a template) to give it to the model as context.

What works well for WordPress development

Task Example prompt
Hooks and snippets “Add a filter that removes WordPress emojis from the head”
Custom post types “Create a ‘project’ CPT with a ‘sector’ taxonomy”
Queries “Write a WP_Query for the 3 latest posts in the SEO category”
Shortcodes “Create a [cta] shortcode with a title and a link as parameters”
Debugging Paste a PHP error from debug.log and ask for an explanation
Redirects “Convert this list of URLs into 301 rules for .htaccess”

What works well for technical SEO

Task Example prompt
Structured data “Generate a LocalBusiness JSON-LD for a restaurant in Madrid”
Search Console regex “Write a regex that filters queries containing quote or price”
Hreflang tags “Generate the FR/EN/ES hreflang tags for this URL”
Processing exports “Write a Python script that lists pages with no meta description in this CSV export”
Bulk tags “Suggest a title under 60 characters for each of these 10 pages”

What I don’t hand over to it

  • Writing articles: Qwen2.5-Coder is specialized in code. Its prose is correct but flat, and it doesn’t know the latest SEO news.
  • SEO strategy: keyword choice, competitor analysis, internal linking across a whole site. It lacks perspective and up-to-date data.
  • Production code without review: I always check data escaping and test on a staging site before deploying.

In short: the local model is my assistant for WordPress code and technical SEO, not for content or strategy.

The limits, and my verdict

A 3B model is not a high-end online assistant. It’s fast and useful, but you need to know what to ask it.

A telling example: I asked for “a function that calculates 20% VAT on a pre-tax price”. It returned a function that calculates the VAT amount, not the price including tax. Correct, but minimal: a small model takes the request literally. You have to be precise in your instructions.

Good for Less good for
Autocomplete, short functions Architecture of a whole project
Explaining a piece of code Reasoning across several files
Regex, scripts, conversions Autonomous agents (Cline and the like)
Working offline and privately Vague or ambiguous requests

To go further on the same machine, you can try qwen2.5-coder:7b. It’s more accurate, but it overflows from VRAM into RAM, so it will be slower.

My verdict: for a developer with a 4 GB graphics card, a local AI is already a real day-to-day gain. It complements an online assistant without replacing it: I keep local for client code and repetitive tasks, and the cloud for complex problems. All for free, in under an hour of setup.