Strata: 12GB VRAM + 32~64GB RAM to run 125B Qwen3.8-Flash-Next

[WhoCares]

Strata is very hot recently.

3-tier: VRAM -> RAM -> SSD

https://github.com/Niko1221/Strata

[chants]

Wow their practical benchmarks are in the 40 to 100 tokens per second range. Could absolutely do some useful work with a rig like that.

NVIDIA RTX 3060 12GB $280
64GB DDR5 (2x32GB) 6000MHz CL30/CL34 $180
AMD Ryzen 5 7600 or Intel i5-12400 $180
1TB NVMe M.2 SSD (Gen 4 preferred) $60
Budget B650 (AMD) or B760 (Intel) $120
650W–750W 80+ Bronze PSU & Budget Case $110

So around $930 to build a new rig and if buying the GPU used maybe even $820. That is not bad at all. I can see why it is hot.

[WhoCares]

It's surprisingly usable on my RTX 4070 + 64GB RAM + 5800X3D, though it's definitely a bit slow.

I need a 256K context window for Claude Code CLI to chew through Strata's own source repo with IQ2_XS. 128K just doesn't cut it.

[WhoCares]

There is a fork for Strata:

https://github.com/zsariboga/Strata
Strata v0.1.43 tuned for a 12 GB laptop (RTX 5070 Ti Laptop, 64 GB RAM): Swift 1.5 IQ3_XXS ~140 tok/s engine decode, vision on. Branch laptop-12gb, report in docs/LAPTOP_12GB_TUNING.md

With the above fork, I can run Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS with 60 token/s, 256k context, 80GB commited memory in Win11.

The abliterated model:
https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/tree/main/IQ3_XXS

[Farooq Ali]

Originally Posted by WhoCares

There is a fork for Strata:

With the above fork, I can run Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS with 60 token/s, 256k context, 80GB commited memory in Win11.

The abliterated model:
https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/tree/main/IQ3_XXS

Any updates on how this is working out in practice?
Is it good for reversing and coding? Or just as a general purpose LLM?

[WhoCares]

I'm using the Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS model. It's a general-purpose model capable of code generation and reverse engineering. Strata also maintains a dedicated code-specialized model.

When running Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS via the DeepSeek harness, complex reasoning workloads can occasionally get stuck in an infinite reasoning loop toward the end: it repeats identical thought trajectories and actions indefinitely, requiring manual prompt swap intervention to break out. Its output token throughput fluctuates — sometimes hitting 60 tokens/sec, other times topping 100+ tokens/sec, root cause unknown.

Originally Posted by Farooq Ali

Any updates on how this is working out in practice?
Is it good for reversing and coding? Or just as a general purpose LLM?

[bolo2002]

Originally Posted by chants

Wow their practical benchmarks are in the 40 to 100 tokens per second range. Could absolutely do some useful work with a rig like that.

NVIDIA RTX 3060 12GB $280
64GB DDR5 (2x32GB) 6000MHz CL30/CL34 $180
AMD Ryzen 5 7600 or Intel i5-12400 $180
1TB NVMe M.2 SSD (Gen 4 preferred) $60
Budget B650 (AMD) or B760 (Intel) $120
650W–750W 80+ Bronze PSU & Budget Case $110

So around $930 to build a new rig and if buying the GPU used maybe even $820. That is not bad at all. I can see why it is hot.

At first, I thought that this post were on the price in 2025.but not still 2026.
where can you find 64GB DDR5 (2x32GB) 6000MHz CL30/CL34 for $180??

[WhoCares]

This Strata fork added a loop breaker in v0.1.44
https://github.com/zsariboga/Strata

STRATA_LOOP_BREAK=1

[chants]

Originally Posted by bolo2002

At first, I thought that this post were on the price in 2025.but not still 2026.
where can you find 64GB DDR5 (2x32GB) 6000MHz CL30/CL34 for $180??

Sorry you are right that was the early 2025 price. Nowadays used is like $700 to $850 and new is $1k to $1.2k. Hope the RAM market makes a recovery with some new AI breakthroughs in memory.

[Farooq Ali]

Originally Posted by WhoCares

I'm using the Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS model. It's a general-purpose model capable of code generation and reverse engineering. Strata also maintains a dedicated code-specialized model.

Thanks, I asked because I had doubts seeing that pagoda model generation pic on their README.
I was interested to know about the real-life RE perfromance. I guess I'll give it a quick try then. I'll use the loop-breaker too.