Havuz

[EMNLP 2026] WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces

HAEnv — synthetic longitudinal patients with code-derived gold, for evaluating health agents over time

MerchantBench is a 365-day, order-level benchmark for evaluating the long-term coherence of LLM agents in seller-side e-commerce operations.

Feel how fast a local LLM runs before you buy the hardware

Agent Memory Benchmark

LLM gateway comparison: self-hosted and managed AI API gateways compared on protocol compatibility, billing units, timeout and retry semanti

DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with

A benchmark for AI-driven CAD generation and editing

Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

📖 Data Engineering from Raw Corpora

aws-bench measures how well AI agents and model combinations perform on real AWS work — diagnosing misconfigurations, provisioning infrastru

[NeurIPS 2026] SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

glTF 2.0 versions of common graphics research scenes

A counterfactual battery for instruction following in vision-language-action policies: change one word of the instruction, hold the scene, a

Real-event-anchored benchmark for detecting AI-generated videos in real-world crisis settings.

Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open model

GateMem: a benchmark and evaluation toolkit for memory governance in multi-principal shared-memory LLM agents.

Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

Benchmark local LLM serving by replaying real agent trajectories

A benchmark for evaluating LLM × harness performance.

The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it,

A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

Ultra-Fast Python Solutions: 110x Speed Optimization Guide 2026

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Anti-slop skills for AI agents: slopbeth cleans the shipped writing, slopgent cleans the conversation.

AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?

SRA-Bench and SR-Agents: a benchmark and toolkit for skill-retrieval-augmented LLM agents.

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity re

Benchmark AI Agents on Enterprise Workflows

State-Aware Collaboration Protocol: auditable work receipts for AI agents.

🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure

Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward-mo

NeurIPS2026 , Scalable annotation pipeline for action-aglined fine-grained instruciton for Visual-language-Action model

JevGym is an open-source platform designed to benchmark and facilitate better probabilistic estimation in Jev-alike models

[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.

ReviewBench is an open, reproducible benchmark for evaluating AI code review systems on real-world pull requests.

Agents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, own

Honest local LLM deployment planning and benchmarking for high unified-memory Macs and future Linux/NVIDIA rigs.

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

NEXBENCH measures what an autonomous agent can actually do on-chain — execute transactions, route swaps, bridge funds, manage DeFi positions

[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.

[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset

Offline cyber security agent for your apps and AI coding tools. Open R&D: measured in public, improved every release.

An agent skill that plays ARC-AGI-3. One rule: say what an action will do before you spend it. Claude Code on Opus 5 finished all 25 public

Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models

Official code for ForesightSafety-VLA (IROS 2026): a unified diagnostic safety benchmark for vision-language-action models on RoboTwin

A benchmark for testing whether coding agents can resolve engineering tasks in scientific software

[NeurIPS 2026] MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare.

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs