LIVE TELEMETRYGPT-5$1.25 / $10.00Sonnet 5.5$2.00 / $10.00Gemini 3.8$0.75 / $3.75DS V4.1$0.15 / $0.60RTX 50901,792 GB/sM4 Max546 GB/sUS Avg$0.182/kWh
TELEMETRY CONSOLE NAVIGATION

Guncelkal Telemetry Index

Search VRAM specs, API costs, or Dev tools

AI & LLM EconomicsQ4_K_M • 30.51 GB Total

Mixtral 8x7B MoE Q4_K_M in Single-GPU 32 GB VRAM

Full 46.7B weights in GDDR7 with only 12.9B active FLOPs per generated token.

Hardware & Model Configuration

16,384 tokens
1 stream
64 GB
KV Cache Precision:
CLI LAUNCH SNIPPET
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 \
  ollama run mixtral:8x7b-instruct-q4_k_m --ctx-size 16384
Total Required
30.51 GB
Safe cap: 30.4 GB
Est. Speed
26.7 tok/s
Decode bandwidth bound
KV Cache
1 GB
Q8_0 @ 16K ctx
Max Context Fit
14,581
Tokens before spill
VRAM ALLOCATION & KV-CACHE MAP30.5 GB / 30.4 GB Usable
Safe Pool (30.4 GB)Physical 32 GB
Weights: 28.31 GB KV Cache: 1 GB Runtime: 1.2 GB
Status: Partial System RAM Offload (Slower Decode)(+0.11 GB spilled past VRAM pool)

All 7 GGUF / Quantization Tiers on NVIDIA RTX 5090 (32 GB GDDR7)

QuantizationMemory NeededSpeed Est.Placement
FP16 / BF16 (16-bit)Reference Lossless95.6 GBOOMOOM
Q8_0 (8-bit)Near-Lossless51.82 GB10 tok/sRAM Offload
Q6_K (6-bit K-Quant)Studio Fidelity40.49 GB15.7 tok/sRAM Offload
Q5_K_M (5-bit Medium)Minimal Loss35.47 GB20.1 tok/sRAM Offload
Q4_K_M (4-bit Sweet Spot)Recommended Default30.51 GB26.7 tok/sRAM Offload
IQ3_XS (3.3-bit I-Quant)Noticeable Degradation22.05 GB229.1 tok/sPure GPU VRAM
IQ2_XXS (2.1-bit Extreme)Emergency Fit Only15.04 GB348.9 tok/sPure GPU VRAM

All 11 Open-Weight LLMs Ranked for NVIDIA RTX 5090 (32 GB GDDR7) (Click Row to Switch)

ModelBest Pure-VRAM QuantMemoryEst. Speed
Llama 3.1 8B InstructUltra-fast RAG extraction, IDE autocomplete & classificationFP1617.96 GB79.6 tok/s
Qwen 2.5 Coder 14BMid-tier local VS Code copilot balancing speed & repo accuracyQ8_018.02 GB81.4 tok/s
Mistral Small 24BClean instruction following on 24 GB single-GPU workstationsQ8_027.65 GB45.2 tok/s
Gemma 2 / 3 27BMultilingual & editorial writing in single 24–32 GB GPUsQ6_K26.08 GB56.8 tok/s
Qwen 2.5 32B / DeepSeek-R1 32BGold-standard local coding & reasoning on RTX 3090/4090/5090 & 48GB MacsQ6_K29.55 GB43.1 tok/s
Mixtral 8x7B MoE (46.7B Total)Sparse Mixture-of-Experts: needs 47B RAM footprint, computes at 13B speedIQ3_XS22.05 GB229.1 tok/s
Llama 3.3 70B / DeepSeek-R1 70BGPT-4 class private local reasoning for 48 GB–128 GB rigsIQ2_XXS23.12 GB65.2 tok/s
DeepSeek R1 671B MoE (Full Frontier)Cluster or Mac Studio Ultra / dual EPYC server deployment (37B active per token)Needs Offload / Cluster409.9 GBOOM
Qwen3-30B-A3B (MoE — 3.3B Active)Ultra-fast 30B Sparse MoE decoding at 3.3B speed; effortless fit on 24 GB cardsQ6_K26.66 GB457.8 tok/s
Qwen3-32B (Dense Dual-Thinking)2026 dense local reasoning & coding benchmark with dual thinking/non-thinking toggleQ6_K29.8 GB42.7 tok/s
Llama 4 Scout 109B (MoE — 17B Active)Multimodal 109B MoE (17B active) built for 64–128 GB Macs, Dual-3090, or H100 podsNeeds Offload / Cluster68.78 GB10.8 tok/s

Telemetry Methodology & Benchmark Sources

Read full equations →

Pre-configured telemetry for Mixtral 8x7B MoE (46.7B Total) (Q4_K_M) on NVIDIA RTX 5090 (32 GB GDDR7). Requires 30.51 GB (28.31 GB weights + 1 GB Q8 KV cache) against 30.4 GB usable pool for ~26.7 tok/s decode.

Primary Technical & US Grid Benchmarks

  • llama.cpp GGUF Model Sizing
  • Apple Metal / CUDA Wired VRAM Limits
  • GQA Attention Context Scaling Spec

Engineering & Pricing FAQ

Can NVIDIA RTX 5090 (32 GB GDDR7) run Mixtral 8x7B MoE (46.7B Total) without CPU RAM offload?
Fits wholly in 32 GB RTX 5090 for blazing ~150+ tok/s MoE output. At Q4_K_M with 16K context, total required memory is 30.51 GB compared to 30.4 GB safe usable GPU VRAM.
What max context length fits inside NVIDIA RTX 5090 (32 GB GDDR7)?
At Q4_K_M quantization on single-stream inference, safe KV-cache capacity supports up to 14,581 tokens before spilling into host system RAM.

Continue in Console Workflow

AI & LLM Economics

Cloud vs. Rig TCO

Plot cumulative 36-month API spending against hourly RunPod/H100 rentals and owned RTX/Mac hardware with electricity and resale value.

Open Workbench
Hardware & Power

kWh Power Cost Lab

Calculate daily, monthly, and annual kWh bills using 80+ PSU efficiency, duty cycle, optional AC cooling load, and 50 US state EIA rates.

Open Workbench
Hardware & Power

PSU Wattage & Spike

Combine dual-GPU draw, CPU peak boost, NVMe arrays, and ATX 3.1 transient spikes to hit the 50–70% PSU efficiency sweet spot.

Open Workbench