Using AI models to train other AI models is becoming a key area of exploration for the next generation of AI laboratories. On the 28th local time, a researcher from Anthropic’s Fellows Program demonstrated what this approach could look like when put into practice.
Anthropic published a new paper, “Automated Researchers Can Reliably Mitigate Alignment Failures,” describing how AI systems can be used to improve model performance across a range of alignment benchmarks. The research team designed tests for 10 specific misalignment behaviors. The automated system ultimately improved all 10 metrics without sacrificing the model’s overall capabilities.
The system was developed under the leadership of Yuehan Chen (phonetic translation), a member of Anthropic’s Fellows Program, and its workflow resembles traditional scientific research. The automated system first reviews existing literature, proposes training methods, trains models according to those plans for 30 minutes, and gradually improves test scores through multiple rounds of experimentation. Effective methods are retained while ineffective ones are discarded, allowing the entire research process to scale rapidly.
Overall, the paper notes, these results provide preliminary evidence that automated alignment post-training could become a genuinely viable approach in the near term.
This research also represents a step toward recursive self-improvement, which many consider the next major phase of AI development. If AI models can improve their own alignment training methods, it may also be possible for them to further improve broader training processes. At that point, human AI researchers might gradually become unnecessary.
The paper also directly compares the performance of automated alignment researchers (AARs) and human researchers: “The best AAR methods took an average of just 6 hours to outperform the approaches proposed by experienced human researchers. Human-guided research directions did not lead to stronger performance.”
The cost gap is equally striking. “AARs require only about $4 per hour in API inference costs (note: approximately 27 yuan at the current exchange rate), while we paid human researchers $150 per hour (approximately 1,011 yuan at the current exchange rate).”
However, this approach still has significant limitations. The effectiveness of the automated system first depends on whether the benchmarks accurately reflect the true alignment objectives. Even when the tests themselves are reliable, creating and maintaining these benchmarks over the long term requires substantial investment. The research literature on which automated researchers rely also requires ongoing maintenance and expansion.
Reference
Automated researchers can reliably mitigate alignment failures