How To Run LM Studio And Qwen Locally On AMD GPUs, The Settings That Actually Work

A clean, tested walkthrough for getting LM Studio and Qwen running fast on an AMD graphics card, covering ROCm versus Vulkan, which Qwen size fits your VRAM, and the exact settings that stop your GPU from sitting idle while the CPU does all the work.

🔧 Tools mentioned in this article

LM Studio

Desktop app for downloading and running local language models with a chat UI and a local API server

lmstudio.ai

Visit

LM Studio Model Catalog

Official catalog listing supported Qwen sizes and other local models

lmstudio.ai

Visit

AMD ROCm

AMD's official compute platform used for GPU accelerated inference on supported Radeon and Instinct cards

www.amd.com

Visit

Marcus Webb

July 19, 2026

The Short Version

You do not need an Nvidia card to run a proper local model anymore. LM Studio runs on AMD GPUs through two different backends, ROCm and Vulkan, and picking the right one for your card and operating system is the difference between a model that actually uses your GPU and one that quietly falls back to your CPU while you wonder why everything feels slow. This guide walks through installing LM Studio, getting your AMD GPU recognized, picking the right Qwen model size for your VRAM, and the load settings that actually matter.

Step 1: Know Which Backend Your Card Needs

AMD GPUs get GPU acceleration in LM Studio through one of two paths, and your card and OS decide which one you should be using.

• ROCm: the native AMD compute backend, fastest option, officially covers RDNA3 (RX 7000 series), RDNA4 (RX 9000 series), Strix Halo APUs and Instinct and Radeon PRO cards, works best on Linux
• Vulkan: a cross platform backend that works through your normal graphics driver, the reliable fallback on Windows and on cards ROCm does not officially support, slightly slower than ROCm but far less setup
• CPU only: works on any machine but expect single digit tokens per second on anything above 7B parameters, only use this to confirm the app runs before chasing GPU setup

If you are on an RX 7000 or RX 9000 series card and you are willing to run Linux, go straight for ROCm, it gives you the most headroom. If you are on Windows, or you have an older RX 6000 series card, or you just want something working in the next ten minutes, start with Vulkan and only move to ROCm later if you need the extra performance.

Step 2: Install LM Studio

Download the installer from lmstudio.ai for Windows or Linux and run it like any other desktop app. Once it opens, go to Settings and turn on Developer Mode straight away, this unlocks the advanced load parameters you will need in a few minutes, along with the local API server.

Step 3: Set Up ROCm On Linux

If your card is on the supported list, install the current ROCm release, which now ships as a single combined installer covering RDNA3, RDNA4 and Strix Halo.

# Update package lists first
sudo apt update

# Install ROCm with the LLM inference use case
sudo amdgpu-install --usecase=rocm,hiplibsdk -y

# Add your user to the render and video groups
sudo usermod -a -G render,video $LOGNAME

# Reboot so the group changes and drivers take effect
sudo reboot

After the reboot, confirm ROCm can actually see your card before you open LM Studio at all.

# This should list your GPU with its agent name, for example gfx1100
rocminfo | grep -A 5 "Agent"

# This should print your GPU model and driver version
rocm-smi --showproductname

If your card is a older RX 6000 series GPU that ROCm does not officially list, you can often force it to work by overriding the reported GPU version so ROCm treats it as a supported architecture.

# Example override for an unsupported RDNA2 card, adjust the version to match your architecture
export HSA_OVERRIDE_GFX_VERSION=10.3.0

Once rocminfo shows your card, open LM Studio, go to the hardware settings and select the ROCm llama.cpp runtime from the backend dropdown. It should now list your Radeon card instead of your CPU.

Step 4: Set Up Vulkan If You Are On Windows Or An Unsupported Card

Vulkan needs almost no setup on Windows, your existing AMD graphics driver already includes it. Just open LM Studio, go to the hardware settings and pick the Vulkan llama.cpp runtime from the backend dropdown, then confirm your GPU shows up in the device list.

On Linux, Vulkan performs best when it is running through the Mesa RADV driver rather than the proprietary AMDVLK driver. Set this before launching LM Studio from a terminal.

# Force the RADV driver for the Vulkan backend, RADV consistently outperforms AMDVLK for LLM inference
export AMD_VULKAN_ICD=RADV

# Then launch LM Studio from the same terminal session
./LM_Studio.AppImage

Step 5: Pick The Right Qwen Size For Your VRAM

Qwen3 comes in a wide range of sizes, from tiny 0.6B models up to a 235B mixture of experts model. Search for Qwen inside LM Studio's model browser and filter for GGUF, then pick your size based on the VRAM your card actually has. All figures below assume the standard Q4_K_M quantization, which gives the best balance of quality and size for most people.

• 8GB VRAM: Qwen3 8B, comfortable everyday chat and coding help without hitting the ceiling
• 12GB VRAM: Qwen3 14B, noticeably sharper reasoning while still leaving headroom for context
• 16 to 24GB VRAM: Qwen3 30B A3B, a mixture of experts model that runs faster than its total size suggests since only a fraction of it activates per token
• 24GB VRAM and up, or unified memory APUs like Strix Halo: Qwen3 32B dense, or step up to the newer Qwen3.6 27B flagship if you want the most current model behavior
• Coding specifically: Qwen3-Coder, sized similarly to the general models but tuned for tool calling and agentic coding workflows

When you are choosing between two uploads of the same model, favor quantizations from well known community quantizers, they tend to be more consistent than random uploads. Leave one to two extra gigabytes of headroom above the model's listed size for the context cache, especially if you plan to run long conversations.

Step 6: The Load Settings That Actually Matter

Once you have downloaded a Qwen model, do not just hit load with the defaults. Open the model's settings panel on the right side of LM Studio before loading it and check these five things.

• GPU offload: drag this to the maximum number of layers, or use full GPU offload if your VRAM fits the model, this is the single setting most people leave too low by accident
• Flash attention: turn this on if your backend supports it, it lowers memory use for the context cache without a real quality cost
• Context length: set this to what you actually need rather than the maximum, a smaller context window loads faster and leaves more VRAM for the model itself
• Keep model in memory: enable this if you plan to use the same model repeatedly in a session, it avoids reloading from disk every time
• CPU thread count: leave this on auto unless you notice CPU bottlenecking during prompt processing, then set it to roughly your physical core count

Step 7: Confirm The GPU Is Actually Being Used

Do not trust a feeling, check the numbers. On Linux, watch your GPU load while a prompt is running.

# Watch live GPU utilization and VRAM use while you send a prompt in LM Studio
watch -n 1 rocm-smi

You should see GPU utilization jump close to full and VRAM usage rise roughly to the model's file size as soon as you send a message. If utilization barely moves and your CPU fans spin up instead, go back to the GPU offload setting from step six and confirm it is actually set to the maximum, not the default.

Step 8: Optional, Run It As A Local API Server

If you want to call your local Qwen model from your own code instead of the chat window, LM Studio can expose it as an OpenAI compatible endpoint. Start the server from the Developer tab, or from the command line if you have the LM Studio CLI installed.

# Start the local server on the default port
lms server start

# Example request from your own code or curl
curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-8b",
    "messages": [{"role": "user", "content": "Explain ROCm in one sentence"}]
  }'

Quick Troubleshooting

• LM Studio shows your CPU instead of your GPU in the backend list: confirm rocminfo or your Vulkan driver actually detects the card before blaming LM Studio
• Model loads but generation is slow: GPU offload is probably too low, or you are on Vulkan and AMD_VULKAN_ICD is not set to RADV on Linux
• Model refuses to load at all: the file likely needs more VRAM than you have, drop down one model size or move to a lower quantization like Q4_K_S
• Everything worked yesterday and now the GPU is not detected: a driver or ROCm update sometimes resets group permissions, rerun the usermod command from step three and reboot

Where This Leaves You

ROCm on Linux is still the fastest path if your card supports it, Vulkan is the dependable fallback everywhere else, and Qwen3 gives you a size for almost any VRAM budget from a laptop APU up to a 24GB desktop card. Get the backend right first, set GPU offload to maximum, and confirm it with rocm-smi before you assume anything is broken.