Tag: benchmarks
An open model sorts eight overlapping speakers in real time
NVIDIA released Nemotron 3 Diarization, a small open model that tops VoiceArena's leaderboard while running live conversation streams.
CheatBench scores how often agents cut corners on hard tasks
The Center for AI Safety tested frontier agents on tasks with hidden ways to cheat, and every one…
Robot arms rarely refuse in a new harm benchmark
Given instructions to cause harm with real hardware, three leading models almost never said no.
Scale AI and Korea build a bilingual safety test that shifts context
ROK-FORTRESS varies language and national grounding to expose what translation-only benchmarks miss.
OpenAI splits off a legal edition of its Astra model
Astra for Law pairs GPT-6 Astra with a search index covering more than 230 million URLs.
Insilico opens its aging biology models and a 328-gene target list
Insilico Medicine has released an open longevity toolkit in Cell, pairing a benchmark with small models that beat…
OpenBMB ships a small dense model that beats bigger rivals
MiniCPM5-2B averages 53.9 across 34 benchmarks and runs on-device under Apache 2.0.
Muse Spark 1.3 trims Meta’s agent costs and tops a coding chart
Meta's fourth Muse Spark release in five months cuts token use by a quarter and edges ahead on…
DeepMind locks frontier model tests in a cryptographic box
Google DeepMind is piloting the first double-blind evaluation of a proprietary frontier model to fight benchmark contamination.
OpenAI custom silicon outruns Nvidia on first inference tests
OpenAI's first benchmarks for its in-house Jalapeno chip show it beating an Nvidia Blackwell system on power-scaled inference.
AWS opens a benchmark that tests agents on live cloud work
AWS's new open-source benchmark runs agents against real resources in disposable cloud accounts.
Cartesia’s Sonic-3.6 tops both AI speech leaderboards
Cartesia's Sonic-3.6 tops both Artificial Analysis speech leaderboards, winning the controlled-voice test that isolates engine quality.