Skip to content
Main Site News Console

Decision 3.0: 0.6B–27B Open-Source Multimodal Decision Models Top Both Jev Decision Index Leaderboards

· Reddit · r/LocalLLaMA Translated
Reddit

We teamed up with the vLLM Semantic Router team to release Decision 3.0 (d3)—our brand-new family of SOTA decision models ranging from 0.6B to 27B parameters. Released under the Apache-2.0 license, all models natively support text, image, and video inputs.

How It Works

  • Built on Qwen3.5 (0.8B / 2B / 4B / 9B) and Qwen3.8-27B. Parameter counts include the vision encoder: 0.33B in the 4B model and 0.46B in the 27B model.

  • Supports three decision types: choice, yes/no, and score. Outputs probability distributions for each answer without requiring text generation. A single call can answer multiple questions for the same input, with each question running an independent forward pass conditioned on the input.

  • Image support: Multiple images per request, with resolutions up to 1.6 MP.

  • Video support: Multiple videos per request (MP4, WebM, MOV, MKV) sampled at 2 fps, up to 32 frames per clip, and up to 0.2 MP per frame.

  • BF16 weights, loadable via Transformers (trust_remote_code=True), trained on AMD Instinct MI325X.

Evaluation Results (evaluated by us on published weights using the official Jev 0.3 toolkit)

  • #1 across both text and vision: The 27B model scored 64.7 on text and 71.6 on vision, outperforming all models on the leaderboard (including Jev’s official models).

  • Class-leading across all sizes: The 0.6B, 0.8B, 4B, and 9B models all lead their respective size classes. Notably, the 9B model (d3-flash) outperforms our previous-generation 27B model (Decision 2.0).

  • Zero-shot video performance: None of the models were trained on video data. Across all 19,140 questions in the Perception Test validation set, the 27B model scored 77.4 and the 4B model scored 70.3.

HF Collection: https://huggingface.co/collections/vllm-sr/decision-30

Top Comments

pmotiveforce (⬆88): These types of models are becoming pure commodities at this point. It’s hilarious that Typesafe AI is somehow valued at $7.5B. Holiday_Point_603 (⬆27): Slap “AI” onto whatever you’re doing and valuations just mysteriously skyrocket anyway. james_pic (⬆10): Pretty sure the secret is just catching the same elevator as a VC. Brilliant-Weekend-68 (⬆27): Seriously though, why on earth is the company behind Jev valued at $7.5B? ArjixGamer (⬆20): Dumb investors. JackMasterN1 (⬆10): So d3-mini 4B just straight up mogs. MikeLPU (⬆16): sglang and llamacpp already support the systemone API. RaGE_Syria (⬆10): Has anyone benchmarked how fast it is? Microsoft’s recently released Decision-1 claims an 85ms response time. Purely for fun, I’d love to try using it for game controls.