UK AI Safety Institute Teams Up With Anthropic
The UK AI Safety Institute announces a landmark partnership with Anthropic to conduct pre-deployment evaluations of fron…
🏷️ Model Evaluation
14 articles about 'Model Evaluation'
Latest AI Models Still Make Three Types of Systematic Reasoning Errors
The ARC Prize Foundation analyzed 160 test runs of OpenAI's and Anthropic's latest models on the ARC-AGI-3 benchmark, id…
CL-bench Life: Can Language Models Learn from Real-Life Scenarios?
A new study introduces CL-bench Life, a benchmark that systematically evaluates the ability of large language models to …
BatteryPass-12K: First Digital Battery Passport Compliance Dataset Released
A research team has released BatteryPass-12K, the first publicly available benchmark dataset for Digital Battery Passpor…
When LLMs Face Retirement: A Production System Model Migration Framework Emerges
A latest arXiv paper proposes a Bayesian statistics-based LLM production model migration framework that enables reliable…
Study Reveals LLMs 'Playing Dumb' Is Positional Collapse, Not Answer Avoidance
A new preregistered study using option-order randomization experiments found that when large language models are prompte…
New Research Corrects Performance Estimation Bias in Imbalanced Classification
A latest arXiv paper proposes a method to correct performance estimation bias for minority class sub-concepts in imbalan…
DenialBench Benchmark Reveals Consciousness Denial Training Across 115 AI Models
A systematic study covering 115 large language models has released the DenialBench benchmark, quantitatively analyzing h…
New Benchmark Tackles the Thorny Problem of LLM Output Determinism
The developer community has launched a new benchmarking tool specifically designed to evaluate whether large language mo…
Don't Rush to Go All-In on DeepSeek V4 — Hear What Industry Practitioners Have to Say First
DeepSeek V4's technical report has sparked industry-wide frenzy, but beyond the impressive specs on paper, 10 frontline …
XTC-Bench: The First Cross-Task Consistency Benchmark Challenging Unified Multimodal Models
Researchers introduce XTC-Bench, the first benchmark to systematically evaluate semantic consistency between visual unde…
When Machine Learning Meets Philosophy: The Negative Ontology of the 'True Target'
A latest arXiv paper reexamines the existential assumption of the 'True Target' in machine learning from a philosophical…