Introduction

We compare Qwen 3.6 27B with its successor, Qwen 3.8 27B, in all six benchmarks of our benchmark suite. Video reviews often describe Qwen 3.8 27B as the better model (see the videos on its model page). Here we check this on our tests.

• Models: Qwen 3.6 27B and Qwen 3.8 27B, dense models of the same size.
• Modes: instruct mode and thinking mode. Qwen 3.8 27B has thinking levels; we ran Think: medium (see the next section). In the charts, "Thinking / Think: medium" means the thinking runs of both models.
• Quantization: this article focuses on NVFP4, because it needs less VRAM and matters more for local users. Every run was repeated with FP8; the section "NVFP4 vs FP8" near the end compares the two.
• Runtime: vLLM, runs from 20 to 24 September 2026.

Every test runs once, and its first answer counts: there are no reruns of failed tests. A model might solve a failed task on a second try, but our aim is to show what a user can expect from a single request, not the best possible result.

Thinking levels in Qwen 3.8 27B

The thinking levels of Qwen 3.8 27B only add a line to the system prompt. They are not a separate mode of the model, and they set no token budget. Qwen 3.6 27B can only switch thinking on or off; Qwen 3.8 27B adds three levels, xhigh, medium and low, chosen with the reasoning_effort parameter of its chat template.

xhigh (the default) adds this line:

Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.

medium adds nothing. The model thinks without any instruction, in the same way as Qwen 3.6 27B.

low adds this line:

Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.

We tested medium for two reasons. First, the default xhigh level is known for very long thinking: video reviewers and users reported long waits for answers, for example in discussion 97 and discussion 113 on the model page. Second, medium adds no instruction, so both models think under the same conditions.

Overall score

In thinking mode, the two models get a similar overall score; in instruct mode, Qwen 3.6 27B scores a few percentage points higher. The overall score comes from our benchmark analysis, which combines the six benchmarks into one number. In thinking mode, the two models are within one percentage point of each other.

Instruct mode

Qwen 3.6 27B Instruct scores higher in four of six benchmarks. In most benchmarks the difference is small; only in IFBench and HumanEval is it a few percentage points larger. Qwen 3.8 27B Instruct scores slightly higher in SC# agentic and LiveCodeBench.

The same results as a difference per benchmark: a bar below zero means that Qwen 3.6 27B scores higher.

Thinking mode

In thinking mode, the two models are within about one and a half percentage points of each other in every benchmark. Qwen 3.8 27B scores slightly higher in LiveCodeBench, Qwen 3.6 27B in HumanEval, IFBench and BBH. Differences this small are within the spread between runs (see the FP8 section), so we read the thinking results as a tie. These results are for the medium thinking level only. The default xhigh level was not tested; with its longer thinking, Qwen 3.8 27B may score higher there.

The same results as a difference per benchmark: a bar below zero means that Qwen 3.6 27B scores higher.

Very hard tasks

On the very hard tasks, the two models differ more than anywhere else: Qwen 3.6 27B solves more of them in BBH, Qwen 3.8 27B slightly more in IFBench. Both benchmarks mark every task with a difficulty level.

In BBH, Qwen 3.6 27B passes more of the very hard tasks, in both modes. This is the largest and most consistent difference in our tests. On the medium and hard tasks, the two models get almost the same score. So the small overall gap in BBH comes almost only from the very hard tasks. The FP8 runs show the same gap.

In IFBench, it is the other way round in thinking mode: Qwen 3.8 27B passes slightly more of the very hard instruction-following tasks. With FP8, the difference on these tasks is larger. This is the only area where Qwen 3.8 27B scores higher in a consistent way.

Answer length

In thinking mode, Qwen 3.8 27B with Think: medium writes fewer output tokens than Qwen 3.6 27B in five of six benchmarks. Only in LiveCodeBench are the two about equal.

Outside LiveCodeBench, Qwen 3.8 27B almost never gave a broken answer in thinking mode. Qwen 3.6 27B did, mainly in IFBench and BBH. In LiveCodeBench, the longest reasoning chains of both models run into the output limit, and the number of such tests changes a lot from run to run.

Shorter reasoning also saves time: on the normalized hardware of our benchmark analysis, Qwen 3.8 27B in thinking mode needs less time per task than Qwen 3.6 27B.

Unwanted tool calls

Qwen 3.8 27B in instruct mode sometimes calls tools that do not exist; Qwen 3.6 27B does not. In HumanEval and LiveCodeBench, the model gets no tools: it only has to write a function. Qwen 3.6 27B never tried to call a tool there, while Qwen 3.8 27B Instruct did, with NVFP4 and with FP8. Most of these calls are empty <tool_call> blocks. Others call tools that do not exist in our tests, such as Read, Write, Bash, WriteFile, submit or mcp__scsharp__create_or_update_file. In thinking mode, this did not happen.

One answer to a LiveCodeBench task shows where such calls may come from:

<tool_call>
<tool_call>
=Read(file_path="/tmp/claude-1000/-home-admin/e7149fbb-bf68-4fb5-aa85-8a7ddbad577b/scratchpad/solve.py)
</tool_call>
=Write(content='#!/usr/bin/env python3
import sys
…

The tool names, the mcp__server__tool naming and the /tmp/claude-…/scratchpad path match Claude Code, an agentic coding tool by Anthropic. The model also wrote the solution in Python instead of SC#. This suggests that Qwen 3.8 was post-trained on agent transcripts that were not fully cleaned. We cannot verify this. One possible reason for such training: every tool call gives the model one more step, because after the result it can think again. This may help long tasks, especially at the xhigh level.

Such calls do not always fail a test, but they cost tokens and break the expected answer format. In an agent that offers real tools, an unexpected call to a tool with a familiar name could also run an action that nobody asked for. Tool use can also get out of hand where tools are part of the task: in one SC# agentic test, Qwen 3.8 27B Instruct called the same health check tool more than a thousand times before it finished.

NVFP4 vs FP8

NVFP4 and FP8 give similar results, and NVFP4 weights need about half the memory. Every run in this article was repeated with FP8. The overall scores of NVFP4 and FP8 differ by about one percentage point for each model and mode, in both directions. In thinking mode, the single benchmarks also stay close. In instruct mode, single benchmarks vary more between the two runs, again in both directions. This looks like normal variation between runs, not a loss caused by NVFP4.

The comparison between the models is the same with FP8: similar in thinking mode, slightly higher scores for Qwen 3.6 27B in instruct mode, more very hard BBH tasks solved by Qwen 3.6 27B, and unwanted tool calls only from Qwen 3.8 27B Instruct (even more of them in HumanEval with FP8).

The VRAM saved with NVFP4 can hold a larger KV cache, for longer contexts or more parallel sessions. For local use, NVFP4 therefore gives a good balance of quality and VRAM for both models.

Recommendation

For both models, NVFP4 is a good choice when VRAM is limited: the quality is close to FP8, and the saved memory can go to the KV cache.

This conclusion is limited to the medium thinking level: the default xhigh level of Qwen 3.8 27B was not tested, and it may change the thinking-mode results. In instruct mode, the thinking level plays no role. Our results do not show the improvement that many video reviews describe. Our benchmarks cover SC# coding, instruction following, tool calling and reasoning. Other workloads, the low thinking level of Qwen 3.8 27B and other runtimes may also give different results.

References

This article is based on our benchmark analysis, on the chat templates of Qwen 3.8 27B and Qwen 3.6 27B, and on the runs below.

The runs are linked by their number.

Benchmark runs with NVFP4 quantization

Benchmark runs with FP8 quantization