This is a follow-up to our first blog post, “The agent says it’s done, but the database begs to differ”. ThinkingBox-Training is now open-sourced on GitHub.
We built ThinkingBox to evaluate agents by comparing the final database state against expected outcomes. We then used this executable signal as a reward function on training tasks held out from ThinkingBox-Bench. For Qwen3.8-27B, 50 GRPO steps increased the average success rate from 51.70% to 60.78%, while the number of tasks solved correctly across all 20 attempts jumped from 38 to 107.
Figure 1. ThinkingBox turns executable checks on the final database state into rewards to post-train agents on tasks outside ThinkingBox-Bench.
This post is a joint collaboration between Microsoft and Hugging Face, with special thanks to Mirco Milletari and Xiang Jiang from Microsoft, Sergio Paniego from Hugging Face, and former intern Zhuochun Li (University of Pittsburgh) for their contributions to co-authoring and reviewing.
We initially built ThinkingBox to catch agents that claim to have finished a task when they actually haven’t.
Across all 507 tasks in the ThinkingBox-Bench benchmark, each one provisions a real backend and a suite of tools for the agent, lets it act freely, and then inspects what actually changed in the database. In our first post, we caught a case where a model marked a customer ticket as resolved, yet the underlying carrier exception remained open. Nine seemingly valid tool calls resulted in a single, erroneous database row.
Then we realized the broader potential of what we had built: for the vast majority of tasks, running a programmatic check against the final database state directly yields a 1 or a 0. That’s not just a grader—it can serve as a reward function.
So, we constructed a training set using ThinkingBox-compliant tasks outside the benchmark and used their executable outcomes as rewards. The hard part was figuring out which tasks were suitable for training the agent: most tasks were either already trivial for the model to solve or completely beyond its current capabilities—neither of which provides any meaningful learning signal.
Difficulty is policy-relative: a task that is…
