Havuz

PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game? (EMNLP 2026 Findings)

Benchmark suite: Claw Code (Rust) vs Claude Code (Node.js)

OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language World Models

MiniMax H3 video generation on 2080Ti 22G (Turing sm_75) - field-tested handbook, workflows & FAQ

DrugTargetWorld: a synthetic biobank for training and benchmarking AI scientists on end-to-end drug target discovery. Paper: arXiv:2610.0955

Placebo-controlled benchmark of the most-starred coding agent skills

[ECCV 2026] Towards Spatial Supersensing in the Wild

Decomposing and measuring evaluation awareness in existing benchmarks and our proposed EvalAwareBench.

Protein Fold Classification at Scale: Benchmarking and Pretraining (ICML 2026 oral)

An open benchmark for what consent banners cost the pages they sit on — time to banner, payload, viewport blocked and page stability, measur

IdeaTaste - A benchmark for research taste: let LLM agents propose research ideas, then check how many of them the research area actually we

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Diagnosing policy grounding in LLM judges via causal erasure. Companion code to the AIES 2026 submission READ.

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

Qwen3.8-27B on RTX 5090 Laptop 24GB — TWIN-TURBO NVFP4 (native llama.cpp) vs Coder390 Q3 LynnStyle (.ninfer + NInfer engine in WSL2/Docker):

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

Bittensor Subnet Repository maintainer agent + a GitHub-history benchmark that scores it — can an agent make the maintainer decisions a stro

Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation

SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation

[NeurIPS 2026 Meta-Agents] SAEScientist-Bench: Can AI agents conduct autonomous SAE interpretability research?

Evaluation code for SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images (NeurIPS 2026).

WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

A benchmark for AI agents as teammates. Paper: https://arxiv.org/abs/2611.02633

Scenarios and tasks for evaluating AI agents on real-world AWS automation tasks. Use the aws-bench framework (https://github.com/aws-bench/a

Caveman prompting, measured. A two-channel evaluation framework for prompt compression and LLM token efficiency: what input vs output compre

Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAG

AcademiClaw: When Students Set Challenges for AI Agents — a bilingual benchmark of 80 university student-sourced academic tasks.

CLI for benchmarking PostgreSQL queries before and after optimization and generating an HTML report.

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors,

LintEval: Benchmarking LLMs on Linter-Guided File Refactoring (EMNLP 2026 Findings) — multi-language benchmark for linter-guided refactoring

OMCBench is a benchmark suite for evaluating malicious-code detection capabilities. The benchmark consists of a labeled set of 800 Python an

Manipulation benchmarks test acting to change; EmbodiedLab tests acting to know. Given only an objective and the apparatus, a robot runs its

Reproducible evaluation for AI coding agents. Multi-turn scenarios against Claude Code, Codex, Copilot, Cursor, Gemini CLI, Goose, OpenCode,

Use many on device and cloud models with Apple's Foundation Models framework in iOS and macOS with a full chat experience.

Precision-aware retrieval benchmark for LLM memory systems.

Generative UI benchmark

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving

A Minesweeper benchmark for LLM agents. One identical board, up to nine models in parallel, one clock, one tool layer.

A 125-task benchmark for knowledge-mediated implicit associations and retrieval blind spots in long-term agent memory.

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Official code for EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents.

Profine automatically profiles and optimizes PyTorch training jobs on real GPUs, delivering measurable speedups and lower GPU costs before t

Benchmark model duels in Doom. An MCP-native arena for real-time model-vs-model evaluations.

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

[NeurIPS 2026 Verify-Agents] SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

The Virtual Filesystem Interface for AI Agents

Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.020

Benchmark aberto de LLMs para o processo legislativo brasileiro

AMD Radeon 9700 AI PRO RDNA multi GPU Tools for llm/vLLM inference, tuning, overclocking, undervolting and LLM models benchmarking