Havuz

A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The ga

RSI-Exam: Measuring Recursive Self-Improvement on Long-Horizon, Executable Research Tasks · Discord https://discord.gg/FBzhepyE

Evaluating frontier models on real-world ai-infra engineering tasks

AI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NP

Benchmarking coding agents on formalized undergraduate research problems

Physical-Engineering’s Last Exam

The Virtual Filesystem Interface for AI Agents

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

DiagSafe: diagnostic LLM safety benchmark with prompt-level rubrics (3,489 bilingual prompts, 84-label taxonomy)

SkillCorpus, dağınık SKILL.md dosyalarını agent'lar için seçilmiş, aranabilir skill corpus'larına dönüştüren ve retrieval ile evaluation araçları sunan açık kaynak bir altyapıdır.

Neden yükseliyor · Agent'lar için paylaşılan ve göreve göre seçilen skill katmanına ilgi var; repo kısa sürede 667 yıldıza ulaşmış ve son 28 günde 9 commit ile 3 farklı yazar katkı vermiş. README ayrıca birden fazla benchmark'ta iyileşme bildiriyor; bu sonuçları bağımsız olarak doğrulayamıyorum.

Evidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results.

[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.

ICML 2026 benchmark and MechVL models for multimodal understanding of mechanical engineering drawings.

Chinese-first LLM jailbreak benchmark & red-team toolkit | 中文LLM安全评测框架 | by POJIA.AI

A consistent, fair, benchmark for structured extraction.

Reproduction package for "Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity" — ten agent arms across AppWorld an

Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

Placebo-controlled benchmark of the most-starred coding agent skills

Manipulation benchmarks test acting to change; EmbodiedLab tests acting to know. Given only an objective and the apparatus, a robot runs its

🚀 Automated scraper & injector for 100+ free AI coding models into 9router combos. Live pre-tested, ranked by coding benchmarks & zero-conf

DolphinBench - Mapping the Pareto frontier of Agent Memory

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

Create your own benchmark for coding AI Agents

NarrowWide: A LiDAR-inertial dataset with repeated transitions between confined and open spaces (IEEE T-FR 2026)

Open-source computational lithography workflow, manufacturability benchmarking, and foundation model data engine for advanced EUV/curvilinea

Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.020

SciPIvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

Ten frontier VLM agents, five robotic capabilities, 1,000 cases. GeoProbe and traceable analyses of geometry, spatial reasoning, affordance,

Reproducible benchmark that compares VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack

SelfStarter, bir coding agent’a görev brief’i verip işi insan yönlendirmesi olmadan birden çok oturum boyunca yürütmesi için tasarlanmış erken aşama bir harness’tir.

Neden yükseliyor · 58 günlük bir repoda 387 yıldız, uzun süre kendi işini yürüten agent’lara yönelik ilgiyle uyumlu. README’deki Agent City örneği de fikri somutlaştırıyor. Ancak son 28 gündeki 1 commit ve 1 farklı yazar, güncel geliştirme temposu veya geniş katkı tabanı için güçlü kanıt sunmuyor.

A benchmark for AI-driven CAD generation and editing

aws-bench measures how well AI agents and model combinations perform on real AWS work — diagnosing misconfigurations, provisioning infrastru

A benchmark for evaluating LLM × harness performance.

Benchmark AI Agents on Enterprise Workflows

Official code for InfoVis: one 9B model for slide image generation and editing, with the VisSlideBench benchmark and 15M+ training records.

RigMark benchmarks local AI the way coding agents actually use it.

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

Red team vs blue team benchmark for scientific paper verification. Plant defects into open access biomedical papers, build gates to catch th

A unified framework for training, fine-tuning, and evaluating World Action Models

Open-source CLI, API, web UI, and reproducible benchmarks for probabilistic AI-generated image detection.

HarnessEval-W: Agentifying the Evaluation of Visual Worlds

Synthetic fraud graph generator for benchmarking graph-based fraud detection models in financial services.

🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents

Real-event-anchored benchmark for detecting AI-generated videos in real-world crisis settings.

State-Aware Collaboration Protocol: auditable work receipts for AI agents.

[COLM'26] SkillLearnBench is the first benchmark for evaluating continual learning methods that automatically generate agent skills.

[NeurIPS 2026] MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare.

A composable framework for reproducible evaluation of long-running agents.