Havuz

A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence

Open-source benchmark for browser AI agents on daily tasks.

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verifi

skilled: open-source agent skill evaluation CLI and MCP server for Codex and Claude Code. Test skills, detect regressions, and improve with

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

Stateful ride-support agent: versioned claims, evidence-backed RAG, durable memory and guarded MCP tools.

Open source evals for physical AI. Run any LLM/VLA on any arm/humanoid against any real/sim benchmark.

Swap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augment

Yao Meta Skill, tekrar kullanılabilir agent skills oluşturma, değerlendirme, paketleme ve yönetişim süreçlerini tek bir araç zincirinde birleştiren Python projesidir.

Neden yükseliyor · Agent skills ve bunların platformlar arasında taşınabilirliği gündemde; proje, üretimin yanı sıra eval ve yönetişim süreçlerini de kapsayan geniş bir kapsam sunuyor. 180 günde 2675 yıldız ilgiye işaret etse de son 28 günde commit ve farklı yazar olmaması güncel geliştirme veya topluluk ivmesi için kanıt sağlamıyor.

Track 42,331+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.

Robot-agent harness with a CLI and local Web console for LIBERO short evaluation

Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward-mo

The Fable Method, coding agent’lar için görevleri sınıflandırma, kanıt toplama, değişiklik yapma ve sonucu doğrulama adımlarını tanımlayan skills ve bunları sınayan eval’ler sunuyor.

Neden yükseliyor · 83 günde 2294 yıldız almış olması dikkat çekici. README’de belirli senaryolara ait sonuçlar, başarısızlıklar ve ham çıktılara bağlantılar sunulması ilgi görmesini açıklayabilir; ancak yükselişin kesin nedeni bilinmiyor. Son 28 günde commit ve farklı yazar etkinliği yok.

Run your real agent with and without your skills, MCPs, and rules. See which ones actually help, and what they cost in tokens. Supports Clau

🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.

Download official Windows ISOs directly from Microsoft's CDN. Web app + CLI tool. No account, no ads, no browser required.

DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with

ParseBench - A Document Parsing Benchmark for AI Agents

A composable framework for reproducible evaluation of long-running agents.

An open-source Python SDK for analyzing, evaluating, and curating agent traces stored in BigQuery. Built on top of the BigQuery Agent Analyt

Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

HarnessEval-W: Agentifying the Evaluation of Visual Worlds

:dog: Beagle is designed to be the unified platform for the broader Recursive Self-Improvement (RSI) ecosystem. Think of it as Hugging Face

Evaluate (lint, review, and security-audit) your AI coding agent harness.

AI Agent / LLM application engineer Chinese interview question bank: 2,038 questions with reference answers

Harness-oriented agent system framework for production-grade LLM agent applications

Benchmarking Agents on Real-World Terminal Tasks

A lightweight, general-purpose harness for tool-using LLM agents, fair benchmark evaluation, harness baselines, and personal assistant workf

SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor.

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Open-source control plane for governed ModelOps — fine-tune, evaluate, promote, and serve models as cost-capped, policy-gated, audited comma

LoopArena, uzun süreli yazılım görevlerinde bir coding agent’ı yönlendiren modelleri ortak koşullarda karşılaştırmak için benchmark ve çalıştırma altyapısı sunuyor.

Neden yükseliyor · Coding agents’ın uzun süreli iş akışlarında nasıl yönlendirileceği güncel bir araştırma konusu; LoopArena bu konuya açık protokol, benchmark sonuçları ve bir arXiv paper ile dikkat çekiyor. Ancak 146 yıldız, 28 günlük repo yaşı ve son 28 günde tek farklı yazar, ilginin henüz yaygın benimsemeye dönüştüğünü göstermiyor.

🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

Litefuse - Agent Observability and Evaluation Platform

A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The ga

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results

研迹 ResearchTrail｜本地 AI Native投资研究工作台。串联证券、组合、证据报告、论点复审与 Today 简报，让研究有据、判断留痕。Python 业务内核 · Electron 桌面 · 可复现 Agent 评测。

PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

Multi-modal benchmark for measuring sensitive-information redaction in human recording data

Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audio ·

Gen AI Evaluation Toolkit on AWS is a flexible, cloud-native accelerator built on AWS serverless architecture that enables comprehensive eva

The experience graph for AI agents. Self-hosted memory, human-reviewed decisions, and outcome-backed datasets for model evaluation and learn

RAG parameter sweep and evaluation toolkit — a practical way to test how different vector databases, embedding models, chunking strategies,

Precision-aware retrieval benchmark for LLM memory systems.

How much does an image model change what you told it to leave alone? Chained-edit benchmark for image models + installable scorer and Claude

Benchmark for repository-level code understanding: 500 open-ended questions from real GitHub Discussions across 50 repositories, graded by a

EvoTrace, Claude Code ve Codex oturumlarını inceleyip eğitim, değerlendirme ve doğrulama için yeniden kullanılabilir varlıklara dönüştürür.

Neden yükseliyor · 39 günde 292 yıldız, coding agent oturumlarından eğitim ve değerlendirme varlığı üretme fikrine ilgi olduğunu gösteriyor; ancak bu ilginin nedeni ölçümlerden anlaşılamıyor. Son 28 günde commit ve farklı yazar olmaması, ilginin devam eden geliştirme veya benimsemeyle desteklendiğine dair kanıt sunmuyor.