Havuz

TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation

Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audio ·

Community benchmark database for running LLMs on Apple Silicon Macs

[ACM MM 2026] Official implementation of Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

Make every model cheaper or better. Zero-config behavioral harness for Claude Code, with a reproducible 408-run benchmark

This dataset provides metadata, official statistics, and official play-by-play annotations for full-length NBA game videos available on YouT

CLI for agent builders to benchmark & chaos test your AI Agents. Text, Voice, Code supported.

Don't trust an Agent. Test it. Open-source lab for agent credit — exams → evidence → explainable, recomputable scores. Built for the day age

AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether

Blind A/B testing for AI agent skills, .cursorrules, CLAUDE.md, and system prompts. Detects placebo and harmful instructions.

WRBench: camera-controlled generation and diagnostic evaluation of video world models.

Advanced DNS benchmark (UDP, DoH, DoT) with scoring, latency tests and detailed reports

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

[NeurIPS 2026 Verify-Agents] SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

Cache-aware frequency sort: a header-only C++20 sort for low-cardinality integer arrays on x86-64.

📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON

[ECCV 2026] Towards Spatial Supersensing in the Wild

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

Scenarios and tasks for evaluating AI agents on real-world AWS automation tasks. Use the aws-bench framework (https://github.com/aws-bench/a

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

A measurement-grounded benchmark for evaluating physical fidelity in physics engines and video world models.

Simplified Technical English for Code - extract, adapt, and enforce ASD-STE100 rules for software documentation. 9-agent pipeline, 59-test b

Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation

A 125-task benchmark for knowledge-mediated implicit associations and retrieval blind spots in long-term agent memory.

Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate

Bittensor Subnet Repository maintainer agent + a GitHub-history benchmark that scores it — can an agent make the maintainer decisions a stro

A reproducible benchmark for bimanual manipulation in simulation and the real world.

Diagnosing policy grounding in LLM judges via causal erasure. Companion code to the AIES 2026 submission READ.

Per-step LLM routing benchmark with 970 static labels, live SWE-bench evaluation, an open data pipeline, and a public leaderboard.

CoderCup — the continuous public benchmark for AI coding agents, refereed by TestSprite

A public reference benchmark for radiology finding-to-report generation.

Multi-modal benchmark for measuring sensitive-information redaction in human recording data

An external memory for LLMs: a bi-temporal knowledge graph with hybrid (keyword + vector) retrieval and a self-improving reflection loop. Be

View planning with multi-turn VLM agents: ViewSuite 6-DoF benchmark on real ScanNet scenes + iterative RL-SFT training

Open browser arena for embodied / VLA stacking benchmarks

Benchmark for repository-level code understanding: 500 open-ended questions from real GitHub Discussions across 50 repositories, graded by a

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

A preservation-aware benchmark for natural-language SVG repair with deterministic binary rewards and complete model traces.

Community-driven benchmark suite for MLX inference engines on Apple Silicon

StealthBench is a benchmark for evaluating whether autonomous agents operate with the operational security (OPSEC) discipline that separates

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-bound

Benchmark and compare Ollama models across local and cloud endpoints with rich, sortable tables.

Official code for the paper: "Multi-User Large Language Model Agents"

A benchmark suite and re-usable collection of differentiable physics solvers.

PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game? (EMNLP 2026 Findings)

curl performance tests

Open-source computational lithography workflow, manufacturability benchmarking, and foundation model data engine for advanced EUV/curvilinea

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu