Anthropic tested Zhipu AI’s open-weight GLM-5.3 model against its own “Mythos” preview. The test measured the ability to generate cyber exploits. GLM-5.3 nearly matched Mythos.
This result shifts the open-weight debate from theory to practice. A publicly available model almost equals a specialized red-teaming tool.
The Red Teaming Benchmark
Anthropic built the Mythos preview to test worst-case scenarios. It represents the outer limit of Claude’s offensive capabilities.
The company expected Mythos to be unique. It is not. Zhipu’s general-purpose model performed nearly as well on exploit generation.
“This confirms our fears about the rapid capabilities of open-weight models,” Anthropic stated.
Key Findings from the Test
- Model tested: Zhipu AI’s GLM-5.3.
- Comparison model: Anthropic’s proprietary Mythos preview.
- Test type: Exploit generation benchmark.
- Result: GLM-5.3 nearly matched Mythos.
The Access Control Problem
The main difference is not capability. It is access.
Mythos is strictly internal. It cannot be downloaded or modified.
GLM-5.3 is an open-weight release. Anyone can download it and remove restrictions.
Access makes the dangerous capability real. A less capable but widely available model can be more harmful than a locked-away supermodel.
Arguments in the Debate
This finding fuels two opposing views.
Critics of open-weight models use this as proof that releases are too risky. They want strict capability thresholds based on tests like this.
Proponents of open-weight models argue that transparency enables safety research. They say restricting access creates a knowledge monopoly.
Anthropic clearly sides with the critics. The company has long warned about open-weight risks. This test gives it hard data to support the warning.
Zhipu AI’s Response
Zhipu AI rejected the alarming interpretation. The company says GLM-5.3 is a general-purpose model and not designed for malicious use.
Zhipu pointed to its own safety alignment work. It argued that capability benchmarks do not predict real-world misuse.
The company also suggested that open models help the global community study AI risks. Transparency and collective oversight are the better path.
Implications for Policy
Regulators now have a concrete case study. The EU AI Act and US Executive Orders try to classify risky models.
This test shows exactly where the line sits. Open-weight models can cross the threshold of dangerous dual-use capability.
Future models will only be more capable. This benchmark is a leading indicator for the policy challenges ahead.
Consequences for Cybersecurity
The exploit generation test is highly relevant to cybersecurity. It measures a concrete offensive capability.
Open availability of this capability changes the threat landscape. Companies and governments must adjust their defensive postures.
The gap between proprietary and open models is closing in a high-stakes domain. The industry must respond to the evidence.
A Warning for the Future
The speed of this development is worrying. GLM-5.3 did not take years to catch up to proprietary models.
If this trend continues, open-weight models will surpass proprietary red-teaming tools. The policy window for action is closing fast.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.