Muse Glimmer 30B Hits 127 tok/s Locally via DFlash Speculative Decoding

Muse Glimmer is gaining attention in local AI communities and drawing comparisons to Qwen's 27B-class models, with early reports putting it ahead specifically for agentic coding workflows. What makes the setup reproducible is that Meta supplies official GGUF files and a paired DFlash drafter, so the full stack—model, speculative decoder, and serving layer—runs on consumer hardware with no proprietary dependencies. For engineers weighing the cost and data-privacy tradeoffs of cloud coding assistants, a locally-served 30B-class model reaching 127 tokens/second on a single RTX 3090 changes the calculus. This is the kind of pipeline architecture approach increasingly driving practical AI gains rather than any single model leap.

The setup chains three open-source tools: llama.cpp as the OpenAI-compatible inference server, DFlash speculative decoding to accelerate generation, and Pi as the agentic coding client. Each component is separately installable, which matters for teams that want to swap layers independently.

Model Files and Disk Footprint

Both files come from the meta-models/Muse-Glimmer-30B-GGUF repository on Hugging Face. The quantized main model, muse-glimmer-30B-kquant-17gb.gguf, is 16.8 GB. The DFlash drafter, dflash-kquant.gguf, adds 1.63 GB. Total cold-storage requirement is just over 18.4 GB—within a 24 GB VRAM budget with room for context. Both files land in a single directory referenced directly by the llama-server command.

llama.cpp Server Configuration and DFlash Speculative Decoding

llama.cpp is built from source with -DGGML_CUDA=ON and -DCMAKE_BUILD_TYPE=Release for the CUDA-accelerated binary. The serving command offloads all layers of both the main model and the drafter to GPU (-ngl all and --spec-draft-ngl all), enables Flash Attention (-fa on), sets a context window of 16,384 tokens, and binds to port 8080. Speculative decoding is activated with --spec-type draft-dflash and --spec-draft-n-max 15, meaning the drafter proposes up to 15 candidate tokens per step for the main model to verify in parallel.

Throughput numbers from initial testing: short-prompt queries came in at around 46 tokens/second; sustained longer coding tasks reached approximately 127 tokens/second. That upper figure is where the DFlash drafter earns its 1.63 GB of VRAM—speculative decoding pays off most when generation runs long and the drafter's predictions compound.

Pi Integration and Agentic Performance

Pi connects to the llama.cpp server at http://localhost:8080/v1. Installing the pi-llama extension (pi install git:github.com/huggingface/pi-llama) is sufficient for auto-detection; Pi reads the models being served and presents them without requiring manual edits to a models.json file. Selecting the muse alias inside Pi surfaces Muse Glimmer through the llama-cpp provider.

The practical agentic test was a full FastAPI task management API: project structure, CRUD endpoints backed by SQLite, input validation, error handling, pytest coverage for all endpoints, requirements.txt, and README.md—with an explicit instruction that the model must run and fix its own tests without prompting the user. Muse Glimmer completed the build in approximately 2 minutes, identified failures autonomously during its test loop, and delivered a passing suite. At 127 tokens/second, the gap between issue identification and fix drops to a few seconds, which is where latency stops being the bottleneck in multi-step debugging.

Where Muse Glimmer showed weakness was single-shot front-end generation: an HTML game it produced did not function correctly, and Qwen3.8-27B was noticeably better for working HTML apps and games. The model's strength is clearly in structured, iterative back-end work.

Broader Signal

An official GGUF release from Meta, a named and separately downloadable speculative decoder, and a sub-20 GB total footprint represent a meaningful shift in what production-viable local coding requires of the hardware stack. RTX 3090, 4090, and 5090 owners can replicate this setup today. The gap to cloud-hosted models like GLM-5.2 is narrowing to the point where the best small language models on Hugging Face are no longer a clear tier below proprietary counterparts for structured agentic tasks. As DFlash matures and context windows extend, routing every coding request through a third-party API—with its associated cost and data-exposure tradeoffs—becomes harder to justify.