Thermal Meltdown: Why Ollama Pins Your CPU at 100°C While Your GPU Idles (And How to Fix It)

amu · July 24, 2026, 11:27am

The business model of selling tokens is like betting on a dead horse in a race. There are many open-source models out there, but finding the right one requires a lot of knowledge and tuning it so that it fits your hardware is another engineering task altogether. But at the end of the day, you get something that performs within ±0.01% of something like GPT-5.4, completely for FREE.

It’s not as simple as clicking a button, though; there are serious traps, like easily overheating your computer. Left untuned, the model spawned 16 CPU threads, and my CPU temperature instantly spiked to 95°C. Run that load continuously for 2 to 10 hours, and you’ll need a new CPU.

Here are the documented steps (along with some background context) on how to configure the model so it fits your hardware perfectly without destroying your computer in minutes.

If you have experimented with local AI agents like AI Agents, or custom tool-calling frameworks, you have likely encountered the strict hardware requirements of modern agentic workflows. To prevent an agent from losing context mid-task during long multi-step reasoning loops, these systems often enforce a hard minimum context window typically 64,000 tokens (64K) or more.

My Testsystem was a Asus Laptop


NVIDIA RTX 3080 (16GB)
AMD Ryzen 9 5900HX with Radeon Graphics 8c/16t
64DDR4 Ram

on ollama, v0.32.1, Gnoppix 26.8-pre nv-cuda setuped

KMD Version: 610.43.02 CUDA UMD Version: 13.3

```

**The 1st Problem I hit was 4k input restriction from the model, doesnt work with agents …
```

So, you modify your Modelfile, crank num_ctx up to 65536, and launch your prompt.

Seconds later, your system fans sound like a jet engine taking off. You open your hardware monitor: CPU temperature is pegged at a dangerous 95°C–100°C with 100% utilization across all cores, while your expensive graphics card sits underutilized at less than 50% GPU load.

Why does increasing the context window turn your CPU into an expensive space heater while leaving your GPU mostly idle? More importantly, how can you fix this bottleneck without buying enterprise-grade datacenter hardware?

Below is a technical breakdown of why this thermal bottleneck occurs, how to diagnose it on your system, and the step-by-step configuration fixes to tame your thermals and unlock your GPU’s full potential.

The Root Cause: The KV Cache Explosion

When running local models via Ollama (which uses llama.cpp under the hood), most developers assume VRAM consumption is determined almost entirely by the model’s parameter size and quantization weight (e.g., a 35B model at Q4_K_M takes up roughly 20 GB of RAM).

However, model weights are only half of the memory equation. The second, often overlooked memory consumer is the Key-Value (KV) Cache.

What is the KV Cache?

During auto-regressive generation, a Transformer model computes “Key” and “Value” attention vectors for every single token in the prompt. To avoid recalculating these vectors quadratically for every new word generated, Ollama stores them in system memory or GPU VRAM—this storage is the KV cache.

While model weights remain static in size regardless of prompt length, the KV cache grows linearly with context length (n\_{\\text{ctx}}) and model depth (n\_{\\text{layers}}):

VRAM\_{\\text{total}} = VRAM\_{\\text{weights}} + VRAM\_{\\text{KV}}

At default 16-bit float precision (fp16), the memory required for the KV cache can be calculated as:

VRAM\_{\\text{KV}} \\approx 2 \\times n\_{\\text{layers}} \\times d\_{\\text{model}} \\times n\_{\\text{ctx}} \\times 2 \\text{ bytes}

Let’s plug in real numbers for a 35B parameter model (such as Qwen 35B with 60 layers and a hidden dimension of 5120):

At 4,096 tokens (Default context): The KV cache consumes ~1.2 GB of VRAM.
At 32,768 tokens (32K context): The KV cache consumes ~9.6 GB of VRAM.
At 65,536 tokens (64K context): The KV cache skyrockets to ~19.2 GB of VRAM!

If your model weights already consume 20 GB of VRAM, adding a 64K context window bumps your total memory requirement from 21.2 GB to over 39 GB of VRAM.

The Hybrid Offloading Cascade

What happens when your GPU only has 24 GB of VRAM (e.g., an NVIDIA RTX 3090 / 4090), but the combined model weights and 64K KV cache demand 39 GB?

Ollama does not simply crash with an Out-Of-Memory (OOM) error. Instead, it performs hybrid offloading. It allocates as many layers as possible onto the GPU VRAM, and silently spills the remaining layers and the expanded KV cache into System Host RAM (DDR4/DDR5).

  +-----------------------------------------------------------------------+
  |                             HOST SYSTEM                               |
  |                                                                       |
  |  System RAM (DDR4/DDR5) <---> CPU (AVX-512 / Vector Math)             |
  |  [ Offloaded Layers (e.g. 20/60) + Overflown 64K KV Cache ]           |
  |  * CPU pegged at 100% utilization (Thermal Limit: 100°C)              |
  +----------------------------------+------------------------------------+
                                     | PCIe Bus Bottleneck
                                     v
  +-----------------------------------------------------------------------+
  |                           GPU ACCELERATOR                             |
  |                                                                       |
  |  VRAM (GDDR6X)                                                        |
  |  [ On-board Layers (e.g. 40/60) ]                                     |
  |  * GPU sits IDLE (<50%) waiting for CPU output on every token loop    |
  +-----------------------------------------------------------------------+

This creates a dual-threat performance bottleneck:

CPU Thermal Throttling: CPUs execute matrix mathematics far less efficiently than GPU tensor cores. When Ollama offloads layers to system memory, it spins up matrix-multiplication loops across every logical thread available on the host processor. Vector instructions (like AVX2 / AVX-512) drive CPU silicon straight to its thermal junction maximum (T\_{j\\text{max}}), reaching 95°C–100°C within seconds.
The GPU Pipeline Stall: Deep learning inference is a synchronous chain. Layer 20 cannot calculate its output until Layer 19 finishes. Because the CPU takes significantly longer to compute its assigned offloaded layers over system RAM, the GPU finishes its on-board layers almost instantly and spends over 50% of its clock cycles sitting completely idle, waiting for the CPU to pass data back over the PCIe bus.

Step 1: Diagnose Your Memory Split

Before changing configurations, inspect how Ollama is currently distributing your model across hardware.

While your model is actively generating text, open a terminal and run:

Bash

ollama ps

Look at the PROCESSOR column:

NAME	SIZE	PROCESSOR
`qwen-agentworld-64k:latest`	38 GB	45%/55% CPU/GPU

If the output reads anything other than 100% GPU, your hardware is spilling over to system RAM and cooking your CPU.

Next, inspect the Ollama log files (or run journalctl -u ollama on Linux) to view the layer distribution:

Plaintext

llm_load_tensors: offloading 38/60 layers to GPU
llm_load_tensors: offloaded 38 paged layers to GPU
llm_load_tensors: 22 layers remaining on CPU host

This confirms that 22 layers are running on your CPU, which explains the 100°C thermal spikes.

Step 2: The Solution Blueprint

To stop CPU offloading, you must squeeze both the model weights and the 64K KV cache entirely inside your GPU VRAM. Here is how to achieve 100% GPU acceleration.

1. Enable Flash Attention & KV Cache Quantization

This is the single most effective optimization for long-context inference.

By default, Ollama stores the KV cache at full 16-bit floating-point precision (f16). However, Ollama supports quantizing the KV cache down to 8-bit (q8_0) or 4-bit (q4_0) precision:

q8_0 (8-bit): Cuts KV cache VRAM usage by 50% with virtually zero impact on perplexity or reasoning accuracy.
q4_0 (4-bit): Cuts KV cache VRAM usage by 75%, reducing a 19.2 GB context cache down to just ~4.8 GB!

Crucial Requirement: KV cache quantization in Ollama only takes effect when Flash Attention is explicitly enabled.

Set these environment variables before starting the Ollama daemon:

Linux (Systemd Service)

Edit the service definition:

Bash

sudo systemctl edit ollama.service

Add the environment variables under the [Service] block:

Ini, TOML

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"

Save, reload, and restart:

Bash

sudo systemctl daemon-reload
sudo systemctl restart ollama

macOS (Terminal / Launchctl)

For a single session:

Bash

export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
ollama serve

For persistent service execution:

Bash

launchctl setenv OLLAMA_FLASH_ATTENTION "1"
launchctl setenv OLLAMA_KV_CACHE_TYPE "q8_0"

Windows (PowerShell)

PowerShell

[System.Environment]::SetEnvironmentVariable('OLLAMA_FLASH_ATTENTION', '1', 'User')
[System.Environment]::SetEnvironmentVariable('OLLAMA_KV_CACHE_TYPE', 'q8_0', 'User')

2. Cap CPU Threads (Thermal Safety Net)

If your model is slightly too large for VRAM and minor CPU offloading is unavoidable, stop Ollama from utilizing 100% of your system’s logical threads.

By default, Ollama assigns inference threads to every available core (e.g., 32 threads on a 16-core CPU), triggering rapid thermal saturation. Restricting execution to your CPU’s physical performance cores (usually 4 to 8 threads) prevents thermal throttling while maintaining identical throughput.

Add num_thread to your custom Modelfile:

Dockerfile

FROM hf.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF:UD-Q2_K_XL

# Set requested context size
PARAMETER num_ctx 65536

# Force all layers to GPU
PARAMETER num_gpu 999

# Cap CPU thread pool to 6 physical cores to prevent 100C thermal spikes
PARAMETER num_thread 6

Rebuild your model container:

Bash

ollama create qwen-agentworld-64k -f Modelfile

3. Choose the Right Model Weight Quantization

If enabling OLLAMA_KV_CACHE_TYPE=q8_0 still leaves your VRAM slightly overfilled, adjust your base model quantization.

When running 64K+ context windows, it is often better to use a slightly more quantized base model (e.g., Q4_K_M or IQ3_XS) so that the entire model fits inside GPU memory alongside the context cache, rather than running a higher-precision model (e.g., Q8_0) that forces 20% of its layers onto the CPU.

100% GPU execution with a Q4_K_M base model will perform 5x–10x faster and run 30°C cooler than a Q8_0 model running in hybrid CPU/GPU mode!

Performance Comparison: Before vs. After

Here is what typical metrics look like on an RTX 4090 (24 GB VRAM) running a 35B parameter model at 64K context before and after applying these optimizations:

Metric	Unoptimized (Default fp16 Cache)	Optimized (q8_0 Cache + FlashAttention)
KV Cache VRAM Usage	~19.2 GB	~9.6 GB (50% reduction)
Layer Offload (`ollama ps`)	38/60 GPU (63%)	60/60 GPU (100%)
CPU Temperature	98°C – 100°C (Thermal Throttling)	45°C – 55°C (Idle/Normal)
CPU Utilization	100% (All threads pinned)	< 5%
GPU Utilization	35% – 48% (Stalled)	95% – 99% (Fully saturated)
Inference Speed	~3.2 tokens/sec	~28.5 tokens/sec

4.) Solved my heat Problem!

Question is now how to fix the heat (3 Options)

Option 1: Power Limit the Ryzen 5900HX (Keep the 35B model)

Since CPU offloading is required, stop the CPU from thermal runaway by capping its Thermal Design Power (TDP).

Tool: Use Universal x86 Tuning Utility (UXTU) or Ryzen Controller.
Setting: Cap the CPU TDP to 25W or 30W.
Result: Max CPU temperatures drop instantly from 95°C to ~72°C–76°C with only a minor ~10% hit to tokens/second.

Option 2: Turn Off CPU Turbo Boost

Disabling aggressive CPU boosting keeps the CPU at base clock (~3.3 GHz instead of 4.6 GHz) during heavy offload loops:

Windows: Set Maximum Processor State in Power Options to 99% (or set Processor Performance Boost Mode to Disabled).
Linux: Run echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost.

Option 3: Switch to a Model That Fits 100% in GPU VRAM

If you want zero CPU offload and full GPU speed:

qwen2.5:14b-instruct-q4_K_M (~9 GB weights + ~2.5 GB 64k KV cache = ~11.5 GB). Fits completely on 12GB/16GB VRAM GPUs with 0% CPU strain.
hermes-3-llama-3.1-8b or qwen2.5:7b (~5 GB weights). Runs purely on GPU with sub-70°C temps.


Option 2 helped for me! Since it is local AI I not really care about time/token usage, using slower CPU on a faster GPU doesn't heat up my computer. Which I'm absolutely fine.

5.0 If your CPU is still hitting 95°C with `num_thread 6`

If your CPU is still hitting 95°C with num_thread 6, it means one of two things is happening: Ollama is spilling layers onto your CPU because of VRAM overflow, or your 5900HX is sharing thermal heat pipes with the RTX 3080 in a laptop/mini-PC chassis.

Here is why this is still happening, followed by three immediate fixes to drop your temperatures.

The Root Cause: The VRAM Math Problem

Your RTX 3080 has either 10 GB or 12 GB VRAM (desktop) or 8 GB or 16 GB VRAM (laptop).

Let’s calculate the physical memory needed for your current setup:

Element	Memory Required
Qwen-35B `Q2_K_XL` Model	~13.5 GB VRAM
64K Context Window (`f16` default)	~19.2 GB VRAM
64K Context Window (`q8_0`)	~9.6 GB VRAM
64K Context Window (`q4_0`)	~4.8 GB VRAM

Total VRAM Required: 23.1 GB to 32.7 GB

Because your RTX 3080 cannot hold 23+ GB, Ollama is forced to offload 30% to 50% of the model layers to your CPU and system RAM, regardless of setting num_gpu 999. That CPU offloading is what is driving your Ryzen 5900HX to 95°C.

Additionally, in laptops or mini-PCs, the CPU and GPU share the same copper heatpipes. When your RTX 3080 runs at 100% load, its heat bleeds directly into the CPU die.

3 Immediate Fixes to Drop Temps Below 80°C

Fix 1: Switch to `q4_0` KV Cache (Saves ~14 GB VRAM)

If you haven’t set OLLAMA_KV_CACHE_TYPE=q4_0 yet, do it now. This shrinks the 64K context memory from 19.2 GB down to ~4.8 GB, bringing total VRAM usage down close to ~18 GB.

Set this environment variable:

Linux / systemd:

Add Environment="OLLAMA_KV_CACHE_TYPE=q4_0" to sudo systemctl edit ollama.service and restart.

Windows PowerShell:

PowerShell

[System.Environment]::SetEnvironmentVariable('OLLAMA_KV_CACHE_TYPE', 'q4_0', 'User')

Fix 2: Drop Threads to `num_thread 4`

On mobile/mini-PC silicon like the Ryzen 9 5900HX, running 6 heavy AVX-512/AVX2 threads pushes the CPU past its thermal dissipation limit. Lowering to 4 threads significantly reduces power consumption and heat output while losing negligible token speed.

Update your Modelfile:

Dockerfile

FROM hf.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF:UD-Q2_K_XL
PARAMETER num_ctx 65536
PARAMETER num_gpu 999
PARAMETER num_thread 4

Rebuild:

Bash

ollama create qwen-agentworld-64k -f Modelfile

Fix 3: Disable CPU Aggressive Boost (The Thermal Circuit Breaker)

AMD Ryzen laptop/mini-PC CPUs aggressively pump high voltage into cores to boost clock speeds, driving temperatures straight to 95°C–100°C (T\_{j\\text{max}}) even under moderate loads.

Disabling Aggressive Core Performance Boost drops CPU temperatures by 15°C to 20°C instantly while keeping base clock speeds fast enough for LLM offloading.

On Windows:

Open PowerShell as Administrator and run this command to reveal the hidden Processor Boost option in Power Options:

PowerShell
```
powercfg -attributes SUB_PROCESSOR 2a373010-6ed6-4350-a577-0921a4f084a2 -ATTRIB_HIDE
```
Open Control Panel \\rightarrow Power Options \\rightarrow Change plan settings \\rightarrow Change advanced power settings.
Expand Processor power management \\rightarrow Processor performance boost mode.
Change it from Aggressive to Disabled or Efficient Aggressive.

On Linux:

Disable CPU boost dynamically:

Bash

echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost

Verification

After applying these fixes, run a prompt and check ollama ps in another terminal.

If your GPU VRAM is still overflowing, you may need to reduce num_ctx slightly (e.g., to 49152 or 32768) to ensure 100% of the model fits on your RTX 3080 without forcing the 5900HX to do matrix math.

Final Verification Checklist

When setting up local agentic frameworks with large context requirements, follow this sanity checklist to keep your hardware running cool and fast:

[ ] Verify OLLAMA_FLASH_ATTENTION=1 is set in your environment.
[ ] Set OLLAMA_KV_CACHE_TYPE=q8_0 (or q4_0 for extremely tight VRAM budgets).
[ ] Set num_thread in your Modelfile to restrict CPU thread saturation.
[ ] Run ollama ps during generation to confirm 100% GPU processing.

By shifting the heavy lifting of context management off the CPU and properly fitting your KV cache inside VRAM, you eliminate thermal bottlenecks, protect your hardware, and drastically boost generation speeds for long-context AI agents.

Thermal Meltdown: Why Ollama Pins Your CPU at 100°C While Your GPU Idles (And How to Fix It)

The Root Cause: The KV Cache Explosion

What is the KV Cache?

The Hybrid Offloading Cascade

Step 1: Diagnose Your Memory Split

Step 2: The Solution Blueprint

1. Enable Flash Attention & KV Cache Quantization

Linux (Systemd Service)

macOS (Terminal / Launchctl)

Windows (PowerShell)

2. Cap CPU Threads (Thermal Safety Net)

3. Choose the Right Model Weight Quantization

Performance Comparison: Before vs. After

4.) Solved my heat Problem!

Question is now how to fix the heat (3 Options)

Option 1: Power Limit the Ryzen 5900HX (Keep the 35B model)

Option 2: Turn Off CPU Turbo Boost

Option 3: Switch to a Model That Fits 100% in GPU VRAM

5.0 If your CPU is still hitting 95°C with num_thread 6

The Root Cause: The VRAM Math Problem

3 Immediate Fixes to Drop Temps Below 80°C

Fix 1: Switch to q4_0 KV Cache (Saves ~14 GB VRAM)

Fix 2: Drop Threads to num_thread 4

Fix 3: Disable CPU Aggressive Boost (The Thermal Circuit Breaker)

On Windows:

On Linux:

Verification

Final Verification Checklist

5.0 If your CPU is still hitting 95°C with `num_thread 6`

Fix 1: Switch to `q4_0` KV Cache (Saves ~14 GB VRAM)

Fix 2: Drop Threads to `num_thread 4`