Articles

34 articles found Showing 30-34

Testing Qwen 3.6 35B for hobby coding and agentic tasks

Qwen 3.6 35B is a fast mixture-of-experts model, and in thinking mode it solves most of our coding and agentic tasks with NVFP4: about 92% in SC# agentic and 93% in HumanEval. Instruct mode much more often ignores information from its context: it writes C# instead of the language described in its prompt, adds explanations to the code and does not stop calling tools. In thinking mode, it is a good choice for hobby coding and learning; for professional use, it fails too often on hard tasks to work without checks.

Qwen 3.6 27B on FinQA: financial benchmark results and dataset fixes

Qwen 3.6 27B (FP8) can solve about 98% of the FinQA financial questions when the tasks are built correctly and graded deterministically, in both instruct and thinking mode. On the original benchmark it passed only about 75% of the questions, a similar share to other open-weight models in a published test. Our review found that about 23% of the original questions have a wrong reference answer, an ambiguous wording or missing data, so most of this gap comes from the benchmark, not from the model. We fixed these tasks in our port of FinQA to the SC# evaluation runtime. Thinking mode gets a similar score to instruct mode, but writes almost three times as many output tokens. The remaining error rate is low but not zero, so the answers still need checking.

Qwen 3.6 35B vs Qwen 3.6 27B: benchmark results analysis

In our six benchmarks with NVFP4, Qwen 3.6 27B scores higher than Qwen 3.6 35B in both modes, most of all on hard tasks. In thinking mode, Qwen 3.6 35B, a mixture-of-experts model, is about four times faster than Qwen 3.6 27B. Qwen 3.6 35B in thinking mode also scores higher than Qwen 3.6 27B in instruct mode. Its instruct mode scores much lower and often gets stuck in tool call loops, which in our assessment makes it unsuitable for agentic use, so Qwen 3.6 35B should be used in thinking mode only.

Qwen 3.8 27B vs Qwen 3.6 27B: benchmark results analysis

In our six benchmarks with NVFP4 and the medium thinking level, Qwen 3.8 27B is no upgrade over Qwen 3.6 27B: in thinking mode the two models score about the same, and in instruct mode Qwen 3.6 27B scores slightly higher in most benchmarks. Qwen 3.8 27B thinks shorter, but in instruct mode it tries to call tools that do not exist. FP8 gives similar results, so NVFP4 is a good balance of quality and VRAM. The default xhigh level was not tested.

Qwen 3.8 Flash Next NVFP4: benchmark results and the broken thinking mode

Qwen 3.8 Flash Next is the best instruct model in our benchmarks, but its thinking mode is broken in our NVFP4 runs on vLLM 0.30.0: answers stop after the first token or the thinking block never closes. Until this is fixed, we recommend not using this setup and using Qwen 3.6 27B or Qwen 3.8 27B in thinking mode: they need far less VRAM and are much better at reasoning tasks.