Havuz

A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence

Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered c

Open-source benchmark for browser AI agents on daily tasks.

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

Open source evals for physical AI. Run any LLM/VLA on any arm/humanoid against any real/sim benchmark.

Robot-agent harness with a CLI and local Web console for LIBERO short evaluation

Track 42,331+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.

Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward-mo

Yao Meta Skill, tekrar kullanılabilir agent skills oluşturma, değerlendirme, paketleme ve yönetişim süreçlerini tek bir araç zincirinde birleştiren Python projesidir.

Neden yükseliyor · Agent skills ve bunların platformlar arasında taşınabilirliği gündemde; proje, üretimin yanı sıra eval ve yönetişim süreçlerini de kapsayan geniş bir kapsam sunuyor. 180 günde 2675 yıldız ilgiye işaret etse de son 28 günde commit ve farklı yazar olmaması güncel geliştirme veya topluluk ivmesi için kanıt sağlamıyor.

ParseBench - A Document Parsing Benchmark for AI Agents

Run your real agent with and without your skills, MCPs, and rules. See which ones actually help, and what they cost in tokens. Supports Clau

🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.

:dog: Beagle is designed to be the unified platform for the broader Recursive Self-Improvement (RSI) ecosystem. Think of it as Hugging Face

The Fable Method, coding agent’lar için görevleri sınıflandırma, kanıt toplama, değişiklik yapma ve sonucu doğrulama adımlarını tanımlayan skills ve bunları sınayan eval’ler sunuyor.

Neden yükseliyor · 83 günde 2294 yıldız almış olması dikkat çekici. README’de belirli senaryolara ait sonuçlar, başarısızlıklar ve ham çıktılara bağlantılar sunulması ilgi görmesini açıklayabilir; ancak yükselişin kesin nedeni bilinmiyor. Son 28 günde commit ve farklı yazar etkinliği yok.

Download official Windows ISOs directly from Microsoft's CDN. Web app + CLI tool. No account, no ads, no browser required.

HarnessEval-W: Agentifying the Evaluation of Visual Worlds

🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

A composable framework for reproducible evaluation of long-running agents.

An open-source Python SDK for analyzing, evaluating, and curating agent traces stored in BigQuery. Built on top of the BigQuery Agent Analyt

Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением

LoopArena, uzun süreli yazılım görevlerinde bir coding agent’ı yönlendiren modelleri ortak koşullarda karşılaştırmak için benchmark ve çalıştırma altyapısı sunuyor.

Neden yükseliyor · Coding agents’ın uzun süreli iş akışlarında nasıl yönlendirileceği güncel bir araştırma konusu; LoopArena bu konuya açık protokol, benchmark sonuçları ve bir arXiv paper ile dikkat çekiyor. Ancak 146 yıldız, 28 günlük repo yaşı ve son 28 günde tek farklı yazar, ilginin henüz yaygın benimsemeye dönüştüğünü göstermiyor.

Litefuse - Agent Observability and Evaluation Platform

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results

Benchmarking Agents on Real-World Terminal Tasks

面向 AI 奥林匹克与认证（IOAI / NOAI / LMCC）的在线评测与竞赛平台：支持客观题、代码题、LLM 工程题与产物提交（类 Kaggle）评测。

OpenRouter TypeScript harness for reproducible LLM benchmarks and evaluations.

SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor.

deedchain, browser agent raporlarındaki iddiaları kaydedilmiş ground truth ile karşılaştırarak rapor doğruluğunu ölçen bir benchmark ve Python kütüphanesidir.

Neden yükseliyor · Repo yalnızca 23 günlük ve 28 yıldızlı; son 28 gün commit ve farklı yazar verisi ölçülmediği için yükselişin arkasındaki aktivite veya benimsenme düzeyi doğrulanamıyor. Rapor doğruluğunu ayrı bir değerlendirme konusu yapması dikkat çekici olsa da README'deki “first benchmark” iddiası bağımsız olarak doğrulanmış değil.

Harness-oriented agent system framework for production-grade LLM agent applications

Unified World Model Inference & Evaluation Infrastructure

GAN-style self-improvement loop for any text artifact: mutate, grade with a SEPARATE model, keep only verified wins (pairwise-judged), rever

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

Open-source Environment toolkit of claw-like agents, support task/harness generation and evaluation

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

A lightweight, general-purpose harness for tool-using LLM agents, fair benchmark evaluation, harness baselines, and personal assistant workf

A design-of-experiments platform for evaluating compound AI systems - find which technique drives quality, by how much, and whether the di

Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

Evaluate (lint, review, and security-audit) your AI coding agent harness.

PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

Compare web access APIs (web unblockers / scrape APIs) head-to-head against a fixed suite of ~90 real-world, hard-to-scrape URLs

Multi-modal benchmark for measuring sensitive-information redaction in human recording data

Reference harness for the Big Finance benchmark of workflow-grounded financial-research questions

The experience graph for AI agents. Self-hosted memory, human-reviewed decisions, and outcome-backed datasets for model evaluation and learn

Gen AI Evaluation Toolkit on AWS is a flexible, cloud-native accelerator built on AWS serverless architecture that enables comprehensive eva

Scientific RL and SFT post-training for AI agents: build evals that can fail, simulate situations, judge checked against people, pass@1 with

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Precision-aware retrieval benchmark for LLM memory systems.

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.