Launched last spring, the Granite Switch project pioneered a modular approach to model development. Today, the Granite Switch 4.1 models come equipped with Decision adapters and the fastest ASR model on the OpenASR Leaderboard, enabling efficient switching between natural language responses, general and specialized decisions, and automatic speech recognition (ASR) tasks—all served from a single hosted endpoint.
Specifically, the Granite-Switch-4.1-8B and -30B configurations include:
-
A new Decision aLoRA for general decision-making (e.g., v1/systemone);
-
A specialized Guardian aLoRA, which on average outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks; and
-
An embedded Granite-Speech-5.0-Turbo-CTC for transcription tasks—the fastest ASR model on the OpenASR Leaderboard.
Combining these three capabilities alongside the underlying base model results in a single model that consumes virtually the same GPU VRAM as running the base model alone. This means delivering far more capability with fewer GPUs.
Furthermore, because Switch adapters are built as cache-efficient Activated LoRAs (aLoRAs), the KV cache generated during base model inference can be reused for any subsequent adapter-based decisions. Paired with our team’s recent contributions to vLLM to accelerate low-rank adapter inference and a collocated ASR encoder, Granite Switch 4.1 models can blisteringly switch between chat requests, decision requests, and speech requests.
Check out our Speech-to-Decisions HF Space, where granite-switch-4.1-8b transcribes audio, generates summaries, screens for toxicity, and evaluates 15 additional user-defined flags in real time—all on a single endpoint.
Granite Libraries: The OG Decision Models
Decision models are not new. Our earlier release of Granite Libraries included a suite of dedicated “decision model” adapters and other task-specific adapters that can be assembled into custom Granite Switch models. While Jev and other closed-source decision models focus on generality, we found that many specialized granitelib adapters often outperform general-purpose models on specific classification tasks.
Compare granitelib adapters here against general-purpose decision models on specialized classification benchmarks.
Because the adapters share the base model’s KV cache, adding specialized adapters on top of a general decision adapter introduces virtually zero overhead. You get the broad coverage of a general model along with the higher accuracy of specialized models for the tasks you care about most.
In a head-to-head race between Granite-Switch-4.1-30B (frozen Granite base + 4 specialized aLoRAs) and Jev-27B (frozen Qwen base + 1 general decision LoRA), Granite-Switch-4.1-30B completed a real-world 5-step RAG workflow significantly faster than its competitor.
Compose and Extend Your Own Granite Switch Model
Granite-Switch-4.1-8B and -30B are “batteries included” releases—Granite Switch models that we pre-compiled with a curated set of practical adapters. But you aren’t limited to what we provide. Anyone can compose their own Granite Switch model, mixing and matching adapters from Granite Libraries to best fit their specific needs.
Looking ahead, we see massive opportunities for the community to build and extend Granite Switch models with custom and third-party adapters. We plan to release an Adapter SDK with tutorials and Granite Switch support for training adapters in Mellea. These adapters can be specialized classifiers that enhance decision-based workflows, or adapters tailored for specialized language tasks. You can even use custom adapters to teach a 3B-parameter version of Granite to play the classic game Doom.
Similar to Jev, our custom 3B Granite Switch model can make split-second decisions to play Doom, but at blistering speeds of 60 to 100 decisions per second, compared to Jev’s reported 10 decisions per second.1 Unlike Jev, this Granite Switch model can also commentate on its gameplay and answer your questions simultaneously.
We look forward to sharing more details and benchmarks on Granite Switch in the coming weeks. In the meantime, we encourage you to experiment with the latest Switch models and check out the resources below to get started.
Resources for Getting Started
-
Run your own aLoRA vs. LoRA race on Colab.
Footnotes
1 https://typesafe.ai/blog/introducing-system-one-models-and-jev
