How Hugging Face Community Evals Works with EvalEval
Every Eval Ever (EEE) and Hugging Face Community Evals are now interoperable. We support cross-publishing and interpretation of evaluation results, while linking open-source models, leaderboards, and a unified store of standardized metadata.
EEE launched in February 2026 as a project of the EvalEval Coalition. It is the first cross-institutional effort to improve how AI evaluation results are presented when reported by first-party and third-party evaluators. Hugging Face also launched Community Evals in February 2026 to decentralize the publication of benchmark scores on the Hub. Together, the two help close the trust, comprehension, and selection gap for users, researchers, and policymakers trying to evaluate models and assessment results.
Evaluation results are how we measure model capabilities, compare models against one another, and reason about safety and governance. But these results are often scattered and hard to compare. They live in papers, leaderboards, blog posts, and evaluation framework logs, all in different formats. Even for the same model on the same benchmark, different executors and settings often produce different scores; for example, LLaMA 65B was once reported as scoring both 63.7 and 48.8 on MMLU. These discrepancies can stem from evaluation settings that we found are commonly left unreported.
EEE addresses the reporting layer. It uses a JSON schema to describe evaluation results, capturing:
- who ran the evaluation
- which model
- how it was accessed
- generation settings
- what the metric actually means
- [recommended] a companion JSONL file for per-sample outputs
Built with feedback from researchers and policy analysts, the schema accepts results from any source, so framework logs, leaderboard scrapes, and numbers from papers are all normalized into the same format. The GitHub repository includes converters, examples, and contribution guidelines. Since launch, the datastore on Hugging Face has grown to around 229,000 evaluation results, covering more than 22,000 models and 2,200 benchmarks, drawn from 31 different reporting formats. Reproducing those runs from scratch could cost hundreds of thousands of dollars, which only reinforces the point: once the data has been generated at such expense, it should not be scattered again.
To learn more about the schema and how to contribute, see here.
Now, it also has better integration and attribution. Contributors can now send EEE results to Hugging Face Community Evals. We built a converter that writes your EEE records into the small YAML files required by Hugging Face, so you do not need to manually maintain the same result in two formats.
This is not just a new feature for existing EEE contributors; it also applies to anyone who publishes or reads evaluation results. People who run first-party evaluations on their own models, as well as third-party evaluations on someone else’s model, can submit to both Community Evals and EEE at the same time. And people browsing on the Hub can see results that are traceable back to the full record. When you submit data through your organization’s official Hugging Face account, your result will be marked as verified on EvalEval, signaling to readers that the numbers come directly from the source. The sections below explain how Community Evals works and what the converter does.
How Hugging Face Community Evals Works with EvalEval
Hugging Face Community Evals has two parts.
A benchmark lives in a dataset repository and is registered by adding eval.yaml. Once registered, that dataset page collects and displays every score reported for it across the Hub as a leaderboard. The list of official benchmarks is also growing over time.
Model scores are stored in .eval_results/*.yaml inside model repositories. They appear on model cards and feed into the corresponding benchmark leaderboard. Results submitted by the model author, as well as results submitted by anyone through a pull request, are aggregated; each score carries a badge indicating whether it was author-submitted, community-submitted, or independently verified. Anyone can add scores for any model by submitting a PR with the correct YAML file, and model authors can close PRs or hide results in their own repositories.
Here is what one of the leaderboards looks like:
Community Evals leaderboard for Humanity’s Last Exam on the Hub
This is where EEE and Community Evals come together. When you send a result to both, two things happen: first, your score appears on the Hugging Face model page and is pulled into the benchmark leaderboard; second, it carries a source badge that links directly back to the full EEE record, which includes the generation configuration, framework version, reproducibility notes, and any instance-level data.
A single evaluation from the EEE datastore (MMLU-Pro), cross-linked at the file level with the Hugging Face model card (b). The source EvalEval badge links to the full JSON record.
These two destinations serve different roles under the same goal. Hugging Face places your result where people look at models, with a link back to the source. EEE preserves the full structured record so the result can be interpreted and, from there, used to support Eval Cards. By sending the data to both, the same evaluation becomes both visible and readable, which is exactly what publishing evaluations is for.
You can see that interoperability below. The same GPQA score shown in the model card above is also rendered in Eval Cards, which combine EEE run data with benchmark and model metadata into a readable record. It is the same evaluation, just presented differently:
How It Works
Hugging Face stores evaluation scores in model repositories

