Qwen3.8-27B on local hardware: what 32 GB actually tells us

Qwen3.8-27B is a dense, multimodal model with a practical problem attached to its name.[1] Twenty-seven billion parameters is small enough to invite local use, but large enough for memory, context length and inference settings to decide whether that use is comfortable or merely possible.[1] This article is an evidence synthesis, not an independent cross-hardware benchmark.[1][3][4]

The model card describes Qwen3.8-27B as a 27-billion-parameter causal language model with a vision encoder.[1] It accepts text, images and video, supports a native context of 262,144 tokens, and can be extended to 1 million tokens with YaRN.[1] The official repository lists the model under the Apache 2.0 licence.[2]

That makes it a useful subject for a hardware comparison.[1] It does not make every benchmark number directly comparable.[1]

A dense model with a hybrid layout

Qwen3.8-27B uses 64 language-model layers.[1] The layout contains 48 Gated DeltaNet linear-attention layers and 16 full-attention layers, with a Multi-Token Prediction head trained for speculative decoding.[1][3]

For this memory analysis, the relevant distinction is that Qwen describes Qwen3.8-27B as dense rather than as a mixture-of-experts model.[1] Qwen3.8-27B does not advertise a small active-parameter count in the manner of a mixture-of-experts model.[1] As a runtime-memory explanation, the weights still have to fit somewhere, while the engine also needs space for recurrent state, attention cache, activations and the requested context.[3] The exact balance is runtime-dependent.[3]

The model thinks by default.[1] The model card documents reasoning_effort values of xhigh, medium and low, as well as a switch for disabling thinking.[1] It also warns that a lower per-turn reasoning setting can increase total latency in an agentic workflow if the model fails more often and has to retry.[1]

That warning is easy to miss in a speed comparison.[1] Qwen’s note implies that a shorter per-turn response is not automatically a faster completed task when retries are included.[1]

What the published scores cover

Qwen reports results across coding, agent tasks, office work, reasoning and vision.[1] The model card gives Qwen3.8-27B a score of 73.0 on Terminal Bench 2.1, 61.7 on SWE-bench Pro, 79.0 on QwenSWEBench and 90.3 on LiveCodeBench v6.[1] These are Qwen-reported model-card results, not local inference measurements.[1]

The same table reports 70.7 on CoWorkBench, 84.3 on OSWorld-Verified, 64.8 on WebArena-Verified and 91.1 on OmniDocBench 1.5.[1] These results describe a model aimed at software tasks, long-running agent workflows, computer use and document understanding, rather than a text-only chatbot.[1]

They also need context.[1] Qwen’s table combines public benchmarks, internal evaluations and results judged by another model.[1] The footnotes specify different harnesses, prompt rules, context windows and sampling settings.[1] One MathVision comparison uses a fixed prompt for Qwen3.8-27B while other models are allowed two prompt variants.[1] QwenSWEBench and CoWorkBench are identified as in-house evaluations, while HLE is judged by GPT-4o.[1]

The scores are therefore useful as a map of the abilities Qwen reports.[1] They are not a substitute for a controlled local test.[1]

What fits on a 32 GB card

SGLang’s deployment documentation lists three relevant checkpoint types for Qwen3.8-27B: BF16, FP8 and NVFP4.[3] It gives approximate weight sizes of 28.5 GB for FP8 and 16.5 GB for NVFP4.[3] Given those documented sizes, the practical inference is that NVFP4 leaves a larger starting margin than FP8 on an RTX 5090 with 32 GB of memory.[3]

The practical implication is not that the margin converts directly into performance.[3] The same documentation describes a hybrid model whose Gated DeltaNet state pool can limit concurrency before the attention KV cache does.[3] It also records separate settings for state precision, speculative decoding and prefill chunk size.[3]

A 32 GB card is a plausible target for some quantised single-user installations, including the RTX 5090 NVFP4 rows documented by SGLang.[3] Those validations use 8,192 input tokens, 1,024 output tokens and concurrency 1.[3] For the no-speculation RTX 5090 NVFP4 row, the page lists --mem-fraction-static 0.90 and --chunked-prefill-size 2048; the NVFP4 checkpoint’s automatic KV setting resolves to FP8, while the page separately varies SSM dtype, speculative mode and the request-length-based --mamba-full-memory-ratio.[3] These are recipe conditions, not universal requirements. That operating point is not a general capacity guarantee: context, vision, MTP, KV-cache type, prompt length and concurrency can change the result.[3][7] A large context window and several simultaneous requests are different workloads, even though they use the same model file.[3]

SGLang recommends NVFP4 for RTX 5090-class hardware in its deployment page, while FP8 is closer to the capacity limit.[3] That is an engine-specific recommendation, not independent evidence that NVFP4 is best for every workload or preserves the same quality in every use.[3]

An independent Hardware Corner test gives the missing context boundary. Using a 16.68 GiB Q4_K Small build, llama.cpp with MTP disabled, full GPU offload and Flash Attention, the test measured about 22 GB at 64K context, 26 GB at 128K and 34 GB at 256K on its RTX 5090 setup.[6] The environment was Ubuntu 24.04, CUDA 12.8, NVIDIA driver 595.58.03 and llama.cpp build 153d324bc.[6]

That result is a measurement of one quantisation and one software stack, not a property of every Qwen3.8-27B installation.[6] It does provide a concrete boundary: native 262,144-token support does not mean that a 32 GB card can sustain the same context with every checkpoint, vision configuration, MTP setting or KV-cache type.[6]

The Unsloth discussion reports a different same-quantisation comparison on an RTX 5090. With UD-Q6_K_XL, q8_0 K/V cache, Flash Attention, --parallel 1 and the llama.cpp server-cuda setup, its bisected ceilings were 208,896 tokens with no vision and no MTP, 131,072 with MTP but no vision, and 65,536 with both vision and MTP.[7] The vision-plus-MTP row used draft n_max=5, and the author reports a multi-slot out-of-memory failure when retained context and a new image shared the pool.[7]

These are configuration boundaries from a community discussion, not a controlled comparison with Hardware Corner.[7] They do, however, show why vision, MTP, KV-cache type and slot count belong in any 32 GB claim.[7]

AMD's local results

AMD reports preliminary Qwen3.8-27B results from llama.cpp with the Vulkan backend and Multi-Token Prediction enabled.[4] The company measured up to 51.8 tokens per second on a Radeon AI PRO R9700 with 32 GB of memory.[4] A Ryzen AI Max+ 395 system reached up to 24.5 tokens per second with 128 GB of system memory and 64 GB assigned as variable graphics memory.[4]

AMD says the figures are averages across at least three runs on Windows.[4] The R9700 test used MTP set to 2.[4] The Ryzen system used MTP set to 4.[4]

These are useful data points because they show two different local designs.[4] AMD’s two headline values put the discrete GPU result higher, but the comparison is not controlled across hardware, operating system, driver, context, prompt, output length, quantisation or MTP window.[4] The Ryzen system uses a much larger shared memory pool, which gives the model more room but does not produce the same throughput in AMD’s test.[4]

The figures should remain labelled as AMD preliminary measurements.[4] AMD identifies Windows 11 Pro 25H2, Adrenalin 26.7.1 and chipset driver 8.05.04.516 in the footnotes, but does not provide a common cross-hardware prompt and context matrix for these two headline values.[4]

The same community discussion also records a llama.cpp vision and MTP failure mode, including repeated slot-position warnings and a reported multi-slot out-of-memory condition.[7] The underlying llama.cpp issue concerns a related Qwen3.6 configuration, so it is a warning about engine and build scope, not proof that every Qwen3.8 installation fails.[9]

When "runs" means different things

The deployment pages make a useful distinction between loading a model, serving a request and measuring throughput.[3] SGLang documents Qwen3.8-27B on H200, RTX PRO 6000, RTX 5090 and DGX Spark.[3] For DGX Spark, the page records boot-and-serve validation at concurrency 1, without a measured concurrency ceiling, throughput result or acceptance-length measurement.[3]

That is a meaningful result, but it answers a narrower question than “is this a good machine for Qwen3.8-27B?” It says the tested configuration started and served.[3] It does not provide a speed figure or a measured concurrency ceiling.[3]

The vLLM recipe supplies a separate counterexample. Its one-RTX-5090 NVFP4 example uses --enforce-eager and a 32K context, while its 262,144-token example uses two RTX 5090 cards under vLLM 0.26.1rc1.dev608+g99a10304d.[8] Those results are not directly comparable with SGLang or llama.cpp, but they show that engine, checkpoint, graph capture and card count can change the practical boundary.[8]

The vLLM-Ascend documentation shows another end of the hardware scale.[5] Its BF16 instructions require a node with eight 96 GB Ascend950DT devices, or multi-device Atlas 800 configurations.[5] Its W8A8 examples also use several 64 GB devices.[5] The page includes a GPQA Diamond reference comparison for BF16, W8A8 and W8A8-MXFP8, but says that performance tuning is still being completed.[5]

The documented cases imply that an open-weight model can be available to download while remaining a multi-device deployment in an unquantised enterprise stack.[5] The vLLM-Ascend page is based on vLLM-Ascend 0.23.0 and says that performance tuning is not fully verified.[5] “Open” describes access to the weights.[2] It does not set a memory budget.[5]

What the hardware evidence says

The current evidence supports a conditional answer, not a universal hardware ranking.[3][4][6]

A quantised Qwen3.8-27B installation is plausible on a 32 GB graphics card, with NVFP4 leaving more room than FP8 on the RTX 5090-class configuration documented by SGLang.[3] AMD reports 51.8 tokens per second on a 32 GB Radeon AI PRO R9700, but that result belongs to AMD’s Windows and Vulkan test setup.[4]

A large-memory system such as the Ryzen AI Max+ 395 configuration can run the model with a larger shared pool, but AMD’s reported generation rate is lower.[4] DGX Spark and Ascend documentation show that the model also has a place in larger unified-memory and multi-device deployments, although those pages do not provide one common throughput scale.[3][5]

The evidence does not justify a universal fastest-hardware claim.[3][6][8] The article’s decision criterion is conditional: choose the quantisation and backend only after deciding whether the workload is a single interactive session, a long-context document task, or a multi-request service.[3][6][8]

A useful test would be smaller than a leaderboard

A reproducible local comparison would need to hold the model revision, quantisation, backend, context, prompt length, output limit, reasoning mode and repetition policy constant.[3][4][6] It should record load memory separately from KV-cache memory, measure prompt processing and generation separately, and report the raw runs rather than one rounded number.[3][4][6]

The test should also include a failure boundary.[6] Hardware Corner measured about 22 GB at 64K and 34 GB at 256K on its RTX 5090 setup.[6] In the same UD-Q6_K_XL table, the Unsloth discussion varies vision and MTP while keeping the quantisation visible, and reports the three ceilings described above.[7] The reported result is the memory boundary under a stated configuration, not a general threshold for useful latency.[6][7] Hardware Corner also reports generation falling from 74.83 tokens per second at 4K to 26.22 at 64K and 22.79 at 128K on that setup.[6] A memory fit therefore does not establish interactive speed at long context.[6]

The 32 GB boundary is therefore a concrete deployment question, not a property that can be inferred from the model’s native context alone.[1][6][8] The article uses that boundary as its decision criterion rather than presenting a single hardware score.[1][3][6]

The sources and deployment pages were checked on 23 August 2026.[1][3][6] Model files, engine builds and documentation are mutable, so the conditions should be rechecked before publication.[1][3][6]