Zhipu AI Releases GLM-5: Claims It is the Strongest Open-Weight Coding Model
The Lede: A new open-weight model has entered the coding arena. Chinese AI company Zhipu AI released GLM-5, claiming on Wednesday that it is the most powerful open-weight model for coding tasks, outperforming major competitors in key benchmarks.
GLM-5 is now available for download via Hugging Face and GitHub under a permissive license. The company asserts the model achieves a 96.1% score on HumanEval, a standard test for code generation. Zhipu AI states its performance surpasses that of OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet on this specific metric.
What GLM-5 Offers
The model is designed to bridge the gap between open-source and closed-source AI capabilities.
- Coding Performance: Zhipu AI claims GLM-5 is the first open-weight model to beat GPT-4o and Claude 3.5 Sonnet on the HumanEval benchmark. On LiveCodeBench, which evaluates real-time coding, it scored 65.9%.
- General Knowledge: Beyond code, the model demonstrated a 96.1% accuracy on the MMLU general knowledge test. This positions it competitively against top-tier commercial models.
- Long Context: GLM-5 supports a context window of up to 128K tokens. This allows it to process substantial amounts of text or code in a single pass.
“We believe that supporting open source is the best way to advance AI technology and drive its adoption,” Zhipu AI stated in its announcement. The company focuses on making advanced AI accessible to developers and researchers.
Benchmark Details and Comparison
Zhipu AI provided specific numbers to back its claims of superior performance.
- Against GPT-4o: GLM-5 scored 96.1% versus GPT-4o’s 90.2% on HumanEval.
- Against Claude 3.5 Sonnet: GLM-5 also beat Claude 3.5 Sonnet, which scored 92% on the same metric.
- Against Qwen2.5-Coder-32B: The model outperformed Alibaba’s Qwen2.5-Coder-32B, which scored 92.7%.
It is important to note that these results are based on the company’s own reported tests. Independent verification of these claims is still pending.
Technical Details and Availability
GLM-5 is a 32B parameter model using the Mixture of Experts (MoE) architecture. This design activates only the most relevant parts of the model for a given task, which improves efficiency.
- License: The model is