Introduction

We compare Qwen 3.6 27B and Qwen 3.6 35B, two models of the same generation with a different design, in all six benchmarks of our benchmark suite.

• Models: Qwen 3.6 27B is a dense model: all its parameters work on every token. Qwen 3.6 35B is a mixture-of-experts (MoE) model: only about 3B of its parameters work on each token, which makes it much faster.
• Modes: instruct mode and thinking mode for both models.
• Quantization: this article focuses on NVFP4, because it needs less VRAM and matters more for local users. Every run was repeated with FP8; the section "NVFP4 vs FP8" near the end compares the two.
• Runtime: vLLM. Most Qwen 3.6 35B NVFP4 runs used vLLM 0.27.1, and the Qwen 3.6 27B NVFP4 runs used vLLM 0.29.0. In four of the six benchmarks, the FP8 runs of both models used the same version (0.27.1).

Every test runs once, and its first answer counts: there are no reruns of failed tests. A model might solve a failed task on a second try, but our aim is to show what a user can expect from a single request, not the best possible result.

Overall score

Qwen 3.6 27B gets a higher overall score in both modes, and the gap is much larger in instruct mode. The overall score comes from our benchmark analysis, which combines the six benchmarks into one number. Qwen 3.6 35B in thinking mode scores higher than Qwen 3.6 27B in instruct mode.

Speed

Qwen 3.6 35B is several times faster than Qwen 3.6 27B, because only a small part of its parameters works on each token. On the normalized hardware of our benchmark analysis, it generates about five times more tokens per second. In thinking mode, it writes longer answers than Qwen 3.6 27B, but it still needs about a quarter of the time per task.

The most useful comparison is across modes: Qwen 3.6 35B in thinking mode needs less time per task than Qwen 3.6 27B in instruct mode, and it also gets a higher overall score. Qwen 3.6 35B in instruct mode is only a little faster than its thinking mode.

Thinking mode

In thinking mode, Qwen 3.6 27B scores higher in five of six benchmarks; only in BBH are the two models equal. The largest gap is in LiveCodeBench. In the other benchmarks, Qwen 3.6 27B scores a few percentage points higher.

The same results as a difference per benchmark: a bar below zero means that Qwen 3.6 27B scores higher.

Hard tasks

The gap grows with the difficulty of the tasks. On easy tasks, the two models score about the same. On hard tasks, Qwen 3.6 27B solves clearly more of them, most of all in SC# agentic and LiveCodeBench. BBH is the exception: there, the two models score about the same at every difficulty level.

Qwen 3.6 35B also gives more broken answers in thinking mode, mainly in LiveCodeBench, IFBench and BBH.

Instruct mode

The instruct mode of Qwen 3.6 35B is weak: it scores lower than its own thinking mode in five of six benchmarks, often by a large margin. Only in BBH are the two modes equal. It also scores lower than Qwen 3.6 27B in instruct mode in all six benchmarks; only in LiveCodeBench and BBH is the difference small.

Since the thinking mode of Qwen 3.6 35B is only a little slower (see "Speed"), its instruct mode brings almost no advantage.

Tool call loops

In the tool calling benchmark (BFCL), Qwen 3.6 35B in instruct mode fails most tests because it does not stop calling tools. Most BFCL tests expect exactly one tool call. Qwen 3.6 35B Instruct often makes the right call first, then goes on calling other tools or the same tool again and again. In thinking mode, the same model usually stops after the right call and passes most tests.

An example: for the request "Search for famous contemporary sculptures in New York", the model first called the right tool, artwork_search_find, with correct arguments. Then it called two other tools, and then tourist_spot_info 496 times in a row, until the test ended.

For an agent, this matters more than the score: a model that does not stop calling tools wastes time and tokens, and it can repeat actions that should run only once.

NVFP4 vs FP8

NVFP4 and FP8 give similar results for both models: the overall scores differ by about one percentage point. The comparison between the models is also the same with FP8: Qwen 3.6 27B scores higher in both modes, the gap is largest in LiveCodeBench, and the instruct mode of Qwen 3.6 35B fails BFCL in the same way.

In SC# agentic, HumanEval, LiveCodeBench and IFBench, the FP8 runs of both models used the same vLLM version. They show similar differences between the models, all in the same direction as with NVFP4, so the different vLLM versions of the NVFP4 runs do not explain the results.

The VRAM saved with NVFP4 can hold a larger KV cache, for longer contexts or more parallel sessions. For local use, NVFP4 therefore gives a good balance of quality and VRAM for both models.

Recommendation

Our benchmarks cover SC# coding, instruction following, tool calling and reasoning. Other workloads and other runtimes may give different results.

References

This article is based on our benchmark analysis and on the runs below. The runs are linked by their number.

Benchmark runs with NVFP4 quantization

Benchmark runs with FP8 quantization