Sakana AI has unveiled a self-improving model that can both solve tasks and judge its own solutions. Over multiple rounds, it gets better at both abilities. But the model cannot catch errors that live in its own blind spots.
The loop: solve, judge, train
The model is trained in a self-improvement loop. It solves a problem, then scores its own solution. The scored results become new training data. Then the model trains on that data and repeats the process.
This setup removes the need for human annotation in every round. The model creates its own feedback.
Solving and judging improve together
The model has two jobs:
- Solver role: Generate answers or solutions for a task.
- Judge role: Check whether a solution is correct, and explain why.
Because both roles are part of the same model, they improve as a pair. Better solutions give the judge better material to evaluate. Better judgments give the solver clearer signals for the next round.
Sakana AI tested this loop on benchmark tasks. Accuracy increased over several cycles. The model also became better at recognizing wrong answers, not just at producing right ones.
The blind spot problem
The model’s improvement has a clear limit. It cannot judge problems that fall outside its own knowledge. If the model holds a false belief, the judge holds the same false belief. There is no outside reference to correct the loop.
This is the key warning in the research:
A self-improving model can polish what it already knows. It cannot discover what it cannot imagine.
The model may appear more capable in later rounds because it produces fewer visible errors. But hidden errors remain. The model’s own judge does not flag them, because the judge shares the same blind spot.
Why self-improvement still matters
The approach is significant for one reason: it lowers the cost of training data. A model that can generate and judge its own examples may need fewer human-labeled answers. That makes large-scale training more practical.
But the blind spot problem means self-improvement is not enough on its own. External checks are still necessary. If a loop runs without any outside validation, it can reinforce bad patterns while looking more confident.
Key takeaways
- Self-improvement can work: The model got better at solving tasks through repeated self-training loops.
- Self-judging also improves: Its ability to evaluate answers became stronger over time.
- Blind spots stay hidden: The model did not catch errors that were built into its own knowledge.
- Human oversight is essential: Outside checkpoints are needed to stop silent mistakes from compounding.
Sakana AI’s model is a useful step toward autonomous learning. It shows that models can become better thinkers by using their own output. It also shows that no model can give itself a truly independent review. Without an external view, self-improvement can only go as far as the model’s own limits allow.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.