🔍 Read the full analysis: 3 Steps To Turn Database Grading Into RL Rewards With ThinkingBox on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub and uses executable checks of final database state as rewards for agent training. In one reported 50-step GRPO run, Qwen3.8-27B’s average success rate improved on a 507-task benchmark, but it solved seven fewer tasks at least once. The results are benchmark findings, not evidence of performance in live production workflows.
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub, using executable checks of an agent’s final database state as reinforcement-learning rewards. In one reported 50-step run, Qwen3.8-27B raised its average success rate on a 507-task benchmark from 51.70% to 60.78%, while the number of tasks it passed in all 20 attempts increased; it solved seven fewer tasks at least once.
The teams trained Qwen3.8-27B using GRPO for 50 steps. They evaluated the resulting checkpoint on 507 benchmark tasks, each attempted 20 times. The authors report that successful attempts rose from 5,242 of 10,140 before training to 6,163 afterward. Those figures correspond to the reported average success rates of 51.70% and 60.78%.
The benchmark’s Observed 20/20 count—the tasks passed in every one of their 20 recorded attempts—increased from 38 tasks, or 7.50%, to 107, or 21.10%. But the share solved at least once, called pass@20 in the source, declined from 453 tasks (89.35%) to 446 (87.97%). The result combines better average performance and more repeatable success on some tasks with slightly reduced task coverage.
For training, the authors say they created a pool of ThinkingBox-format practice tasks separate from the evaluation benchmark. They estimated each task’s success probability with the base model, selected tasks using what they describe as offline policy-relative difficulty, and kept that pool fixed during training. At the end of a task, executable checks inspect whether expected records are present in the backend and provide the reward. The teams say the method scores the required outcome rather than a specific sequence of tool calls.
More Consistent Database Outcomes
For agents that can change business records, a convincing response or plausible series of tool calls does not prove that the requested action was completed correctly. A check of the final database state can score whether the required records actually exist and whether the resulting state matches the task’s expected outcome. Using such checks during training gives the model feedback tied to the outcome, rather than only to how it acted along the way.
The reported results matter because the benchmark shows a shift in both average success and repeated performance: the average rate rose, and more tasks passed all 20 trials. But the lower pass@20 count cautions against describing the run as an across-the-board improvement. Teams considering the approach would need to assess both dependable completion of known workflows and the range of tasks an agent can handle. The reported benchmark alone does not establish that the method improves reliability in production.
The authors also provide a hotel-booking example involving a room downgrade that required manager approval. They say that before training, the agent changed the booking and issued refunds without approval in 15 of 20 attempts. After training, it preserved the booking and payment state in all 20 attempts, with expected records matching in 19. This is an illustration of one task, not a general measure of performance across hotel bookings.
reinforcement learning training tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From Database Grading to Training
ThinkingBox was described as an evaluation approach in which tasks run against a backend with a set of MCP tools and a simulated user with private context. Its benchmark contains 507 tasks. Once an agent finishes, executable checks compare the database’s final state with the records expected for that task. According to the authors, that lets the grader focus on whether the relevant records are correct instead of requiring one prescribed path of tool use.
The training report extends that grading approach into post-training. Rather than using the 507 evaluation tasks as the practice pool, the teams say they selected separate ThinkingBox-format tasks and trained on them before evaluating on the benchmark. The authors characterize the evaluation as out of distribution because its domains, databases and most MCP tools were not encountered during training. The supplied account does not give additional detail on the composition of the task pool or the full evaluation setup.
“For most tasks, an executable check can return 1 or 0 directly from the final database state. That is not just a grader. This can be a reward function.”
— Microsoft and Hugging Face, in their joint post
As an affiliate, we earn on qualifying purchases.
Limits of the Reported Run
The supplied report describes one 50-step run. It does not provide enough information to judge how results might change with different random seeds, training-pool choices or run lengths, or to compare fully against alternative reward designs and training methods. The authors report the seven-task reduction in pass@20, but the material does not establish why coverage fell.
It is also unclear whether the reported changes would carry over to live production workflows, other models or task sets. The source provides no production results or account of training costs. While the teams describe the benchmark as out of distribution, that characterization does not by itself establish broad generalization. Independent reproduction and broader evaluation would be needed to determine how robust the gains are.
As an affiliate, we earn on qualifying purchases.
Reproduction and Broader Testing
With the stack available on GitHub, researchers and developers can inspect the implementation and attempt their own runs. The next useful evidence would include reproduced results across multiple runs, models and training pools, along with clearer comparisons against other reward methods. That would help establish whether the gains depend on this particular run or reflect a repeatable effect.
Before deployment, teams would also need to test workflows that match their operating environment, including cases where approvals are required and where an apparently successful interaction could leave incorrect records. Evaluations should track both repeated success and task coverage, since the reported figures moved in different directions on those measures. The available source does not specify a next release date or a schedule for additional results.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is ThinkingBox-Training?
It is a training stack from Microsoft and Hugging Face that uses executable checks of an agent’s final database state as rewards during post-training. The teams say the stack is available on GitHub.
What changed in the reported Qwen3.8-27B results?
After a 50-step GRPO run, average success on the 507-task benchmark rose from 51.70% to 60.78%. The number of tasks passed in all 20 attempts rose from 38 to 107, while the number solved at least once fell from 453 to 446.
Does the result show improved performance in production?
No production result is provided in the supplied report. The findings are from a benchmark evaluation, and performance in live workflows remains unconfirmed.
Why use final database state as a reward?
A final-state check can determine whether expected records are present after an agent acts. The teams say this lets training reward the required outcome without requiring a particular sequence of tool calls.
What is still uncertain about the results?
The source describes one run and does not establish how results vary across random seeds, training setups, models or live deployments. It also does not explain the reported decline in tasks solved at least once.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
