Skip to content
Main Site News Console

GLM5.3 Flash "Strata" Now Available to Select Users! Project Maya

· Reddit · r/LocalLLaMA Translated
Reddit

I stumbled across this project. Previously, running GLM-5.3-Flash got me only around 10 tok/s, which was basically unusable. Switching to Maya brought it up to 30 tok/s. Even at such a low quant level, GLM-5.3 currently feels much better than Qwen 3.8 Flash Next. Still testing.

This is super exciting—it makes running GLM-5.3-Flash viable for so many more people.

https://github.com/mw00/project-maya

Project Maya

Run a 321B Parameter AI Model on Your Own GPU

GLM-5.3-Flash, private and fast · 1 to 16 NVIDIA GPUs · AMD & Windows (experimental) · Chat, Vision, OpenAI- & Anthropic-compatible API

Models of this scale typically require a data center. Maya lets you run GLM-5.3-Flash (by zai-org, MIT licensed) on hardware you already own: 321B total parameters, ~18B active parameters per token, and up to a 1M token context window. Its inference engine is purpose-built specifically for this model. It keeps the most frequently used experts in GPU VRAM, secondary ones in system RAM, and the rest on NVMe SSD, dynamically swapping them during conversation. It benchmarks your GPU, CPU, RAM, and SSD performance to auto-tune specifically for your setup. All data stays local and never leaves your machine.

  • Blazing fast: Up to 118 tokens/s on 4x RTX 4090s, and 30+ tokens/s on a single RTX 5090. Real user benchmarks below.
  • Close to the original: Maya’s custom quantization retains 97.7% to 99.2% of the FP8 model’s zero-shot accuracy.
  • Drops right into existing tools: Compatible with OpenAI and Anthropic API formats, supports tool calling, and works out of the box with coding agents.
  • Ready to use: Includes a browser-based chat UI with live monitoring, vision input, thinking level controls, and single-command deployment.

Top Comments

mrgreatheart (⬆57): Awesome. This is exactly why I upgraded to an Epyc with 256GB of RAM. I’m loving this trend of dedicated, hyper-optimized engines built for a single model.

JockY (⬆27): I’ve been thinking about this more broadly: now that we can run models capable of long-horizon coding tasks, my entire software stack has become customized—I run “my own” forks of practically everything. Just this weekend, I wanted to run embeddinggemma-2 on pure CPU. I have a 9965 with 192 cores, so I left GLM 5.3 running overnight via omp to coordinate and execute a whole battery of benchmarks, tuning a custom eg2 embedding server across different strategies specifically for this service…

Solary_Kryptic (⬆18): The mainstream hasn’t caught onto this wave simply because very few people on Earth can afford a Threadripper, lol.

inthesearchof (OP) (⬆5): Exactly. Just look at how slow llama.cpp is to optimize popular open-source models—it feels like ollama but 3x slower. I run custom forks for all my models now, and it gives me a 200% to 300% speedup.

inthesearchof (OP) (⬆1): That’s what I was running before. I’ve deleted it since. I have AVX-512.

inthesearchof (OP) (⬆2): Check out the GitHub repo—your setup was practically made for it.

ColoredTriangles (⬆38): > Up to 30+ tokens/s on a single RTX 5090. Real user benchmarks below. Clicks link: IQ2_S. Are you guys just trolling at this point?

bani (⬆17): When are the mods finally going to mandate putting the quant in the post title? It’s long overdue.