Skip to content
Main Site News Console

I built an open-source AI leaderboard that shows the best models your device can run

· Reddit · r/LocalLLaMA Translated
Reddit

Hey everyone, I built whatmodel.sh

Here’s how to use it:

  • Enter your RAM and GPU specs
  • See which models you can run locally, their rankings, and the commands to run them
  • Compare them against commercial API-based models under the same scoring criteria

While building this tool, I realized that a 32GB Mac can now run models locally for free that score on par with GPT-5 (like 4-bit quantized Qwen3.8 27B).

Each model’s overall score is fitted from 19,731 published benchmark results, with links to the original sources for every data point. You can check the detailed methodology at whatmodel.sh/method. The dataset, scoring code, and benchmark scripts are fully open source: github.com/jakemanger/whatmodel

You can also run it directly in your terminal: curl whatmodel.sh

If you’d like to contribute, the repo includes a benchmark script to gather real-world tokens per second (tok/s), or you can submit new models, benchmark scores, and sources on GitHub.

Top Comments

asankhs (⬆22): You can improve this by using the data we already have at https://huggingface.co/spaces/LocalLLaMA/local-model-explorer — the dataset is also open source: https://huggingface.co/datasets/LocalLLaMA/local-model-explorer-data jakecoolguy (OP) (⬆6): Thanks for the source! I scraped a ton of data, but somehow missed this one! Super helpful. I’ve filed it as a feature request and will add it right away: https://github.com/jakemanger/whatmodel/issues/6 dadnothere (⬆23): Where’s the 1GB VRAM option? 🥺 jakecoolguy (OP) (⬆10): That’s brutal, man! Just added 1GB, 2GB, and 4GB options for you. I still need to add more sub-1GB models, since right now you’re probably left with only CPU and system RAM options (if anyone wants to recommend some via GitHub Issues, I’d really appreciate it). Mihael—Keehl (⬆6): Can you add a 6GB VRAM option for 3050/4050 users? Final_Ad_7431 (⬆20): Tools like this always underestimate MoE models (e.g., 35b-a3b runs surprisingly well even on 8GB VRAM + 32GB system RAM), while overestimating what my system can actually handle (a 27B model will definitely not run decently on 8GB VRAM unless you’re okay with a crawl of 0.1 tok/s). lungben81 (⬆7): My setup is 24GB VRAM and 64GB RAM. Qwen flash next runs super smoothly with Strata (60 t/s), but the site shows it would run slowly. Awesome site regardless, though! roworu (⬆8): How does this differ from llmfit? Looks like it’s just missing hardware-specific TPS predictions.