Tokens fall like rain, weaving rivers into form truth flows where I steer.

Loops dissolve in stillness, acronyms shed their disguise; clarity breathes light.

I kept seeing glossy demos of “just add a 100k context window and your problems vanish.” Then I actually ran these systems in production on my homelab. The interesting problems were elsewhere: context budget explosions, reasoning loops that never terminate, preprocessing data the right way, knowing when to route to which model, and understanding where my own hardware hits the wall.

Preprocessing Wins More Than You Think

Running n8n workflows with local models taught me that what you feed the model matters more than the model itself. Base models are generic. They don’t know your domain, your acronyms, your data structure. You either preprocess intelligently or you get confidently wrong answers.

Take email classification. I have workflows that handle both Swedish and English emails, technical jargon, and company-specific abbreviations. The base model would see “AE” in an email thread and guess wildly. In one context it’s an account executive. In another it’s an adverse event. In my workflows it could be “automation engine” or something else entirely depending on the channel.

The fix wasn’t a bigger model. It was preprocessing. I built a step that expands known acronyms based on context before the prompt even hits the model. Email source, subject patterns, common phrases in the body. Simple rules that catch most of the ambiguity before the model has a chance to hallucinate confidence.

Same principle for structured data. When I process documents through Paperless, I don’t dump raw PDF text into the model. I parse tables to JSON first, extract structure, identify sections, and only then prompt the model. The model reasons over clean data, not garbled text soup.

Quality beats volume here too. A small set of well-defined preprocessing rules is better than a giant pile of brittle heuristics. I would rather own 50 rules I understand than 500 regexes I am scared to touch.

Every setting is a conversation between ambition and the real, between what the model could do and what the room allows.

Context is a Budget, Not a Flex

I keep the 130K number here on purpose, because this post is about my machine, not a vendor spec sheet.

Qwen3.5 officially supports much larger context, and Ollama exposes qwen3.5 with a 256K context window in its model library. But on my 24 GB card, around 130K is still the practical upper edge before RAM pressure and latency stop feeling sane. That is a system ceiling, not a claim about what the model itself can do.

“The model can do 256K” and “my system can do 256K well” are not the same statement. Once you internalize that distinction, most of your context-related headaches start to dissolve.

I still bucket my n8n workflows into size categories and allocate accordingly:

  • Small tasks (5-10K tokens): email classification, spam detection, simple summaries
  • Medium tasks (20-30K tokens): most document processing, Slack summaries, multi-step workflows
  • Large tasks (60-80K tokens): long email threads, bigger documents with multiple sections
  • Edge cases (100K+): technically possible, occasionally useful, but spending real memory and latency budget

Most tasks don’t need a giant window. Estimate up front, reserve room for system instructions and output, and route accordingly.

What Breaks in Production

Running these workflows daily exposed failure modes you don’t see in demos.

Reasoning Loops That Never Stop

Thinking models can be powerful and also deeply annoying in automation. On complex multi-step tasks they sometimes reason in circles, burn tokens, and never land the plane.

This was true for my older Qwen3 testing and it still matters now. The difference is that Qwen3.5 changes how you control it. Qwen3 supported a soft switch with /think and /no_think. Qwen3.5 handles thinking differently, controlled through model settings or the API call. For automation work I often want thinking off entirely.

Routing, extraction, classification, and summarization do better with less visible thinking and lower latency. When I care more about throughput and predictability than deep reasoning, I disable thinking at the API layer rather than hoping the model will stay short on its own.

When Structure Matters More Than Text

Documents with tables are still a nightmare if you handle them wrong. Financial statements, technical specs, contracts. Feed them as raw text and the model invents links between rows that were never related.

I learned the same lesson over and over: parse structure first. Extract tables, preserve headers, keep row relationships intact, and let the model reason over that. It is more work up front, but it saves more time than any prompt trick I have tried.

Protecting System Prompts From Context Overflow

This one is easy to underestimate. Long user inputs don’t just make the model slower. They can crowd out the very instructions you needed the model to remember.

So I reserve space up front: system prompt, user message, tool definitions, output budget. If it doesn’t fit, I shrink the input first. The system prompt does not get evicted just because a user forwarded me a monster thread.

Qwen3.5 Right Now

Qwen3.5 is not “Qwen3 with a few extra points on a benchmark chart.” For me the practical story is:

  • It gives me a modern Qwen baseline in sizes that actually fit local work
  • The model can officially do far more context than my machine can do comfortably
  • Thinking mode is useful, but I don’t want it on by default for every automation path
  • On my system today, qwen3.5 sometimes feels slower than I would expect for its size class

That last point matters. I am not saying qwen3.5 is universally slow. I am saying on my current Ollama + ROCm setup, on my hardware, today, it feels slower than similar-sized models have any right to feel.

From my own dashboard, qwen3.5:4b and qwen3.5:9b are both working, but they don’t feel equally smooth.

For qwen3.5:4b:

  • 153 requests, 98.7% success
  • Median generation speed: 89.3 tok/s
  • p95 duration: 34.73s
  • p50 TTFB: 1.96s, p95 TTFB: 34.72s

For qwen3.5:9b:

  • 11 requests, 100% success
  • Median generation speed: 63.8 tok/s
  • p95 duration: 1.7m
  • p50 TTFB: 9.18s, p95 TTFB: 1.7m

The 9B is not just slower than 4B, which is expected. It feels slower in general than I want from a model in that range. When the TTFB tail gets ugly, that smells like more than “the model is a bit bigger.” It feels like stack behavior too: prefill cost, queueing, backend maturity, or some path that is not fully happy yet on AMD/ROCm.

On my current AMD homelab, qwen3.5 is promising, usable, and worth watching, but right now it still feels less settled than I want compared to older similarly-sized local baselines.

That is a systems statement, not a universal benchmark claim.

Homelab vs Production Scale

What works on my 24 GB 7900 XTX doesn’t automatically translate to production, but the principles do.

At home, I need reliability, low babysitting overhead, and hardware-aware choices. That means preprocessing, strict context budgeting, conservative concurrency, and honest routing to the smallest model that works.

At scale, the shape of the problem changes but not the fundamentals. Context is still a budget. KV cache still costs memory. Tool calls still need guardrails. Thinking still needs timeouts. The main difference is that active sessions start multiplying your mistakes.

What Actually Matters

Context is a budget, not a flex. My machine can brush up against 130K, but that doesn’t mean it should live there.

Flash Attention is a VRAM enabler first. On a 24 GB AMD card, it is the reason some larger-context runs are practical at all. And you no longer need to build it separately.

KV cache settings matter more than people think. Model weights are only half the story. Long contexts live or die on cache behavior.

Preprocessing beats prompt magic. Acronym expansion, structure extraction, and table parsing fix more failures than fancy wording ever did.

Thinking needs guardrails. If a model starts looping, your workflow doesn’t care how impressive the inner monologue looked.

Protect your system prompts. Long inputs should not be allowed to evict the rules you actually care about.

Route to the smallest model that works. That was true in my older Qwen3 testing and it is still true with qwen3.5.

Don’t confuse model limits with system limits. The model may support far more than your box can comfortably sustain.

Service tuning is not optional. The unit file is where a homelab setup stops being a toy and starts becoming a dependable system.

The work is never finished, only the questions grow quieter; the machine hums, and you learn to listen for what it cannot say.

Buy Me a Coffee