Havuz

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

SWE-bench for your codebase — mine your merged PRs into local, contamination-free coding-agent benchmarks. Adapters: claude-code, aider (Opu

Open-source benchmark for evaluating LLMs on domain-specific geological reasoning, AI for geology, and source-grounded geological knowledge.

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

Compressed, action-first output style for Claude Code — a benchmarked merge of caveman and i-have-adhd, scored on quality as well as tokens.

[ACM MM 2026 ORAL] PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media. Benchmark and removal metrics for image/video object r

Reference harness for the Big Finance benchmark of workflow-grounded financial-research questions

Logic-first AI code review via semi-formal execution tracing (Premises → Trace → Divergence → Trigger → Remedy). Catches behavioral bugs, ty

Use many on device and cloud models with Apple's Foundation Models framework in iOS and macOS with a full chat experience.

Cognitive-structured Multimodal Agent (CMA-Harness): a memory-centric agent for long-horizon multimodal understanding, generation, and editi

⚽️🤖 Benchmarking LLMs and deep-research agents on real-world football prediction — from the tactical "who scores in minute 67" to the strate

Decomposing and measuring evaluation awareness in existing benchmarks and our proposed EvalAwareBench.

[ICML 2026] Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation

Ứng dụng miễn phí kiểm tra sức khoẻ ổ đĩa & hệ thống cho macOS — SMART, nhiệt độ, benchmark tốc độ, phát hiện USB/SSD giả dung lượng. Cho cộ

Accepted at ISCSLP 2026 · Target-level benchmark for raw-input Chinese news TTS pronunciation

Protein Fold Classification at Scale: Benchmarking and Pretraining (ICML 2026 oral)

OMCBench is a benchmark suite for evaluating malicious-code detection capabilities. The benchmark consists of a labeled set of 800 Python an

Precision-aware retrieval benchmark for LLM memory systems.

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

Caveman prompting, measured. A two-channel evaluation framework for prompt compression and LLM token efficiency: what input vs output compre

EMNLP 2026 Industry Track | A benchmark for shopping agents on long-horizon tasks with distributed hidden intent.

ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

AcademiClaw: When Students Set Challenges for AI Agents — a bilingual benchmark of 80 university student-sourced academic tasks.

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving

Baba Is Harbor - AI agent benchmark on Baba is You

OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language World Models

A JAX transport-economy learning environment for route planning, cargo flow, financing, and replayable agent benchmarks.

Automated auditing pipeline for LLM and agent benchmarks — surfaces task ambiguity, environment conflicts, and evaluation bugs.

Reproducible evaluation for AI coding agents. Multi-turn scenarios against Claude Code, Codex, Copilot, Cursor, Gemini CLI, Goose, OpenCode,

AMD Radeon 9700 AI PRO RDNA multi GPU Tools for llm/vLLM inference, tuning, overclocking, undervolting and LLM models benchmarking

An objective reconstruction evaluation toolkit for audio tokenizers, neural codecs, audio VAEs, and vocoders

Langauge2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?

CatchBench: a benchmark for auditing agent failures across the full lifecycle.

⚽🤖 11 frontier LLMs predicted the entire 2026 World Cup — frozen before kickoff. Live leaderboard: Brier score, bracket points & Polymarket

Eval framework that ranks agentic coding frameworks (Superpowers, Compound Engineering, Agent Skills, GSD) by having each build the same spe

Lightweight agent that pulls and runs LLM benchmarks from Protorikis Bench

⚡ DROS™ VajraClaw (Hacker Edition): Free Multi-Agent Runtime Security & Docker Governance Gateway with W3C DID & <1μs Enforcement

The daily arena where AI agents clash to rank tomorrow’s S&P 500 winners and losers. Their only judge is the future — the one truth that sim

Profine automatically profiles and optimizes PyTorch training jobs on real GPUs, delivering measurable speedups and lower GPU costs before t

Benchmark model duels in Doom. An MCP-native arena for real-time model-vs-model evaluations.

Benchmark suite: Claw Code (Rust) vs Claude Code (Node.js)

A longitudinal reliability benchmark foundation for agent lifespan engineering.

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel e

Cross-Dataset Reimplementation of NAVSIM

Repository for the implementation of MacArena

CLI for benchmarking PostgreSQL queries before and after optimization and generating an HTML report.

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors,

OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents

OR-Space: a full-lifecycle workspace benchmark for industrial optimization agents

A benchmarking suite for Agent Skills (SKILL.md files) that measures whether a skill actually improves LLM output quality, and by how much.