Qwen3.5 Thinking Mode Parameters: Complete Configuration Guide for Deep Reasoning

Mar 24, 2026

The Problem with Qwen3.5 Outputs

I was getting inconsistent outputs from Qwen3.5. Sometimes the model would produce brilliant, step-by-step reasoning. Other times it would jump straight to conclusions—sometimes wrong ones. The same prompt, the same temperature setting, wildly different results.

The issue wasn’t the model. It was my understanding of Qwen3.5’s hybrid inference architecture. Qwen3.5 introduces two distinct modes: thinking mode (deep reasoning with chain-of-thought) and non-thinking mode (fast, direct responses). Using the wrong parameters for each mode was the root cause of my frustration.

Understanding the Two Modes

Qwen3.5’s thinking mode enables extended chain-of-thought reasoning. The model “thinks out loud,” exploring multiple approaches before settling on an answer. This is ideal for complex problems but uses more tokens and time.

Non-thinking mode skips the intermediate reasoning and goes straight to the output. This is faster and more efficient for straightforward queries but lacks the depth for complex problems.

The problem? I was using the same temperature and sampling parameters for both modes. That’s like driving a car in first gear on the highway—you can do it, but it’s not optimal.

Thinking Mode Parameter Configuration

For thinking mode, the key insight is that lower temperature produces more deterministic reasoning chains, while higher temperature allows creative exploration.

| Use Case        | temp | top_p | top_k | min_p | presence_penalty |
|-----------------|------|-------|-------|-------|------------------|
| Precise coding  | 0.6  | 0.95  | 20    | 0.0   | 0.0              |
| General tasks   | 1.0  | 0.95  | 20    | 0.0   | 1.5              |

For coding tasks, I use temperature=0.6. This produces consistent, reproducible reasoning chains. The model follows a logical path without veering off into creative tangents.

For general tasks where I want the model to explore possibilities, I use temperature=1.0. The higher presence penalty (1.5) prevents the model from repeating the same reasoning patterns.

Here’s how to invoke thinking mode with llama.cpp:

# Precise coding task
./llama-cli -hf unsloth/Qwen3.5-35B-A3B-GGUF:MXFP4_MOE \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --ctx-size 32768

# Creative/exploratory task
./llama-cli -hf unsloth/Qwen3.5-35B-A3B-GGUF:MXFP4_MOE \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --presence-penalty 1.5 \
  --ctx-size 32768

Non-Thinking Mode Parameter Configuration

Non-thinking mode requires different settings. Since the model skips reasoning steps, you need parameters that balance speed with output quality.

| Use Case          | temp | top_p | top_k | min_p | presence_penalty |
|-------------------|------|-------|-------|-------|------------------|
| General tasks     | 0.7  | 0.8   | 20    | 0.0   | 1.5              |
| Reasoning tasks   | 1.0  | 0.95  | 20    | 0.0   | 1.5              |

The key difference is the top_p value. I use 0.8 for general tasks, which constrains the output to more likely tokens. This produces focused, concise responses without the wandering that higher temperature can cause.

For tasks that still need some reasoning but don’t require full chain-of-thought, I use temperature=1.0 with top_p=0.95.

To explicitly disable thinking mode:

# General Q&A
./llama-cli -hf unsloth/Qwen3.5-35B-A3B-GGUF:MXFP4_MOE \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --presence-penalty 1.5 \
  --chat-template-kwargs '{"enable_thinking": false}' \
  --ctx-size 32768

The critical flag is --chat-template-kwargs '{"enable_thinking": false}'. This explicitly tells the model to skip chain-of-thought reasoning.

Python API Configuration

If you’re using Qwen3.5 through an OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="local")

# Thinking mode - complex reasoning
def thinking_mode_query(prompt: str, coding: bool = False):
    return client.chat.completions.create(
        model="unsloth/Qwen3.5-35B",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.6 if coding else 1.0,
        top_p=0.95,
        top_k=20,
        presence_penalty=0.0 if coding else 1.5,
        extra_body={"enable_thinking": True}
    )

# Non-thinking mode - quick Q&A
def fast_mode_query(prompt: str):
    return client.chat.completions.create(
        model="unsloth/Qwen3.5-35B",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7,
        top_p=0.8,
        top_k=20,
        presence_penalty=1.5,
        extra_body={"enable_thinking": False}
    )

The extra_body parameter passes the thinking mode flag through the OpenAI-compatible API.

When to Use Each Mode

This was the missing piece in my understanding. I needed a decision framework, not just parameter tables.

Use Thinking Mode when:

Multi-step mathematical or logical reasoning
Code generation with complex requirements
Debugging where you need to understand the model’s reasoning path
Problems that benefit from exploration of multiple approaches
Any task where the “why” matters as much as the “what”

Use Non-Thinking Mode when:

Simple factual queries (“What is the capital of France?”)
Straightforward code transformations
Quick formatting or text manipulation
Conversational exchanges where speed matters
Tasks where reasoning tokens would be wasted

Memory Considerations for Context Size

One critical detail the documentation mentions: “Opening large context consumes significant memory. For 24GB VRAM, recommend —ctx-size 16384.”

| VRAM   | Recommended ctx-size | Notes                           |
|--------|---------------------|----------------------------------|
| 16GB   | 8192               | Basic usage                      |
| 24GB   | 16384              | Sweet spot for most tasks        |
| 32GB   | 32768              | Extended reasoning sessions       |
| 48GB+  | 65536              | Full context for complex tasks   |

The maximum context window is 262,144 tokens, but don’t set this blindly. I tried running with full context on my 24GB card and watched OOM errors consume my evening. Start with 16384 and increase only if you hit context limits.

# Conservative start
./llama-cli -hf unsloth/Qwen3.5-35B-A3B-GGUF:MXFP4_MOE \
  --ctx-size 16384 \
  --temp 0.6 \
  --top-p 0.95

# Only increase if you see context overflow errors
# Check your VRAM usage first: nvidia-smi

Common Mistakes I Made

Using thinking mode for everything: I was wasting tokens and time on simple queries. A “hello” doesn’t need chain-of-thought reasoning.
Using non-thinking mode for complex problems: When I did switch modes, I sometimes forgot to switch back. Complex code reviews need the reasoning depth.
Setting ctx-size too high: I assumed more context was better. It’s not—it just causes OOM errors if you don’t have the VRAM.
Not adjusting presence_penalty: For creative tasks, I was getting repetitive output. Adding presence_penalty=1.5 fixed this.
Mixing parameters across modes: I had one “optimized” parameter set I used everywhere. Each mode needs its own configuration.

Summary

Qwen3.5’s hybrid inference requires mode-specific parameters. Use thinking mode with temperature=0.6 for coding and temperature=1.0 for creative tasks. Use non-thinking mode with temperature=0.7 and top_p=0.8 for balanced responses. Always set ctx-size appropriately for your available memory—24GB VRAM works well with 16384 tokens.

The key insight is that these aren’t just arbitrary numbers. Lower temperature in thinking mode produces more deterministic reasoning chains, which is what you want for coding. Higher temperature with presence penalty allows creative exploration without repetition. Non-thinking mode with moderate temperature and lower top_p produces focused, efficient output.

Configure for your use case, not for the model. The difference in output quality is substantial.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!