Muse Spark Open is an open-weight model released by Meta. This Mixture of Experts (MoE) model brings together text, image, audio, and video into an ~816 billion (816B) parameter model with ~42 billion (42B) activated parameters per token and a context window of up to 1 million (1M) tokens. It supports text generation, providing developers with a unified interface to query across different input modalities. Following Muse Glimmer, Muse Spark Open introduces a larger Mixture of Experts architecture and stands as the most capable open-weight model from the US to date.
Meta trained Muse Spark Open with a focus on agentic coding: reading codebases, writing and debugging code, and calling tools. It also accepts image, audio, and video inputs, allowing the same model to review screenshots, listen to voice memos, or watch screen recordings while working with code. In this post, we’ll show you how to run the model, fine-tune it, and explore what we built with it.
Overall Capabilities and Architecture
Meta describes Muse Spark Open as a code-optimized model trained on real-world developer workflows, such as multi-file refactoring and extended debugging sessions. As an agent-first model, Muse Spark Open can plan, act, observe, and self-correct across long-horizon tasks, including web browsing with images.
| Feature | Muse Spark Open |
|---|---|
| Model Size | ~816B total parameters; ~42B active per token |
| Expert Routing | 256 experts per MoE layer; 8 selected per token |
| Inputs | Text, images, audio, and video |
| Outputs | Text |
| Context Window | Up to 1M tokens |
| Checkpoint Formats | BF16 and NVFP4 |
The checkpoint repository is Muse-Spark-Open-816B-