Havuz

Phone and word timestamp accuracy on human-labeled TIMIT and Buckeye, clean and under reverb, noise, music and babble. Two tracks, forced al

Laya (open source, local) vs TypeSafe Jev (API): two AI models race in Snake and fight in a Mortal-Kombat-style arena. Every move is a real

🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Repro

Compare web access APIs (web unblockers / scrape APIs) head-to-head against a fixed suite of ~90 real-world, hard-to-scrape URLs

Benchmark and compare Ollama models across local and cloud endpoints with rich, sortable tables.

A public reference benchmark for radiology finding-to-report generation.

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Multi-modal benchmark for measuring sensitive-information redaction in human recording data

Make every model cheaper or better. Zero-config behavioral harness for Claude Code, with a reproducible 408-run benchmark

A robot-oriented full-stack framework for long-horizon embodied agents in evolving physical and social environments.

Daily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜

View planning with multi-turn VLM agents: ViewSuite 6-DoF benchmark on real ScanNet scenes + iterative RL-SFT training

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

Beyond Time Shifts (ECCV 2026): a reference-free audio-visual synchronization evaluator built on Qwen2.5-Omni-3B, with SynthSync data, R-GRP

TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation

The same HTTP API in 10 languages at 3 sizes, 30 implementations. Token cost for LLM agents, plus cold start, latency, throughput, CPU and m

A benchmark suite for differentiable physics solvers.

RigMark benchmarks local AI the way coding agents actually use it.

NarrowWide: A LiDAR-inertial dataset with repeated transitions between confined and open spaces (IEEE T-FR 2026)

SWE-bench for your codebase — mine your merged PRs into local, contamination-free coding-agent benchmarks. Adapters: claude-code, aider (Opu

WRBench: camera-controlled generation and diagnostic evaluation of video world models.

Give Vercel AI SDK agents a real persistent filesystem with Amazon S3 Files.

Benchmarking coding agents on formalized undergraduate research problems

Logic-first AI code review via semi-formal execution tracing (Premises → Trace → Divergence → Trigger → Remedy). Catches behavioral bugs, ty

Community-driven benchmark suite for MLX inference engines on Apple Silicon

HoloGeo: Evidence-driven VLM for robust image geo-localization and landmark-bias mitigation.

AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether

⚽️🤖 Benchmarking LLMs and deep-research agents on real-world football prediction — from the tactical "who scores in minute 67" to the strate

A preservation-aware benchmark for natural-language SVG repair with deterministic binary rewards and complete model traces.

[ACM MM 2026] Official implementation of Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

Real-time AI model power rankings (水墨战力榜) — usage, benchmarks, price-performance & speed from OpenRouter, as an ink-wash Streamlit dashboard

Open-source computational lithography workflow, manufacturability benchmarking, and foundation model data engine for advanced EUV/curvilinea

A reproducible benchmark for bimanual manipulation in simulation and the real world.

Reference harness for the Big Finance benchmark of workflow-grounded financial-research questions

CoderCup — the continuous public benchmark for AI coding agents, refereed by TestSprite

Advanced DNS benchmark (UDP, DoH, DoT) with scoring, latency tests and detailed reports

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

An external memory for LLMs: a bi-temporal knowledge graph with hybrid (keyword + vector) retrieval and a self-improving reflection loop. Be

Interaction Bench: Benchmarked on what users feel

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor.

[ACM MM 2026 ORAL] PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media. Benchmark and removal metrics for image/video object r

PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game? (EMNLP 2026 Findings)

Benchmark suite: Claw Code (Rust) vs Claude Code (Node.js)

OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language World Models

MiniMax H3 video generation on 2080Ti 22G (Turing sm_75) - field-tested handbook, workflows & FAQ

Placebo-controlled benchmark of the most-starred coding agent skills

[ECCV 2026] Towards Spatial Supersensing in the Wild

Langauge2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?

Decomposing and measuring evaluation awareness in existing benchmarks and our proposed EvalAwareBench.