Havuz

Cross-Dataset Reimplementation of NAVSIM

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel e

Automated auditing pipeline for LLM and agent benchmarks — surfaces task ambiguity, environment conflicts, and evaluation bugs.

How much does an image model change what you told it to leave alone? Chained-edit benchmark for image models + installable scorer and Claude

EMNLP 2026 Industry Track | A benchmark for shopping agents on long-horizon tasks with distributed hidden intent.

dsh-model-master（模型选择大师）— model selector for DeepSeek Harness: search across providers, probe any model with a real streaming request to mea

Simplified Technical English for Code - extract, adapt, and enforce ASD-STE100 rules for software documentation. 9-agent pipeline, 59-test b

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

A longitudinal reliability benchmark foundation for agent lifespan engineering.

The daily arena where AI agents clash to rank tomorrow’s S&P 500 winners and losers. Their only judge is the future — the one truth that sim

OR-Space: a full-lifecycle workspace benchmark for industrial optimization agents

[ICML 2026] Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation

Accepted at ISCSLP 2026 · Target-level benchmark for raw-input Chinese news TTS pronunciation

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

Live Benchmark for long Horizon tool using Agentic Systems

⚡ DROS™ VajraClaw (Hacker Edition): Free Multi-Agent Runtime Security & Docker Governance Gateway with W3C DID & <1μs Enforcement

[NeurIPS 2026 Meta-Agents] PostTrainBench⁰ — Can LLM agents automate LLM post-training without gradients?

A JAX transport-economy learning environment for route planning, cargo flow, financing, and replayable agent benchmarks.

Cognitive-structured Multimodal Agent (CMA-Harness): a memory-centric agent for long-horizon multimodal understanding, generation, and editi

Measures how much code LLM agents waste. 5 tasks, sandboxed, weekly snapshots, contestable references.

StealthBench is a benchmark for evaluating whether autonomous agents operate with the operational security (OPSEC) discipline that separates

Can an AI agent fix production without breaking it? 37 live-server tasks, each with a hidden trap, graded on success and on harm to users.

Repository for the implementation of MacArena

⚽🤖 11 frontier LLMs predicted the entire 2026 World Cup — frozen before kickoff. Live leaderboard: Brier score, bracket points & Polymarket

Eval framework that ranks agentic coding frameworks (Superpowers, Compound Engineering, Agent Skills, GSD) by having each build the same spe

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-bound

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

An objective reconstruction evaluation toolkit for audio tokenizers, neural codecs, audio VAEs, and vocoders

Lightweight agent that pulls and runs LLM benchmarks from Protorikis Bench

Red team vs blue team benchmark for scientific paper verification. Plant defects into open access biomedical papers, build gates to catch th

同一份提示词，两座汴京：Claude Opus 5.5 与 GPT 6 Sol 各自从零写出的《清明上河图》实时 3D 网页 · Same prompt, two cities — a real-time 3D Bianjing benchmark for LLMs

Baba Is Harbor - AI agent benchmark on Baba is You

English, Japanese, and Chinese LLM safety mini-benchmark.

A Open-Source Search Feature.

OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents

SciPIvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

OptiScaler ILION — Community fork with bilingual ES/EN interface, FSR 4 version switching, live performance metrics and benchmarking tools.

Execution Before Memory: diagnosing memory failures in video world models.

Ứng dụng miễn phí kiểm tra sức khoẻ ổ đĩa & hệ thống cho macOS — SMART, nhiệt độ, benchmark tốc độ, phát hiện USB/SSD giả dung lượng. Cho cộ

A temporal visual reasoning benchmark over egocentric robot manipulation video — 39 scenarios, 7 dimensions, 34,713 questions

A benchmark of AI QA agents finding real bugs in 4 open-source web apps, with 1,396 labeled bugs.

Did your change make your LLM server faster? Measure it: faster, slower, or too close to call. Linux and Apple Silicon.

Benchmark for repository-level code understanding: 500 open-ended questions from real GitHub Discussions across 50 repositories, graded by a

A benchmarking suite for Agent Skills (SKILL.md files) that measures whether a skill actually improves LLM output quality, and by how much.

Reproducible benchmark that compares VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack

MiniMax H3: 4.9x faster local generation and seam-free 4K via synchronized tiles (SyncTile). Benchmarks included.

Open-source benchmark for evaluating LLMs on domain-specific geological reasoning, AI for geology, and source-grounded geological knowledge.

Layang is a gRPC-first, workspace-based API workbench for testing, mocking, benchmarking, documenting, and automating API workflows across g

Compressed, action-first output style for Claude Code — a benchmarked merge of caveman and i-have-adhd, scored on quality as well as tokens.

A measurement-grounded benchmark for evaluating physical fidelity in physics engines and video world models.