Havuz

An audit agent that proves whether a person's data is really gone: resolves identity across systems, judges state at the audit cutoff, finds

Agent skill for web typography: Bringhurst applied to modern CSS, with claims measured in the browser and a baseline eval suite

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs tha

An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark a

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which tra

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifac

Self-hosted evidence store for coding agents.

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.

项目验证工作台：架构可视化、CI 检查、覆盖率分析与修复验证 | A project verification workspace for architecture maps, CI checks, coverage analysis, and evidence-backed f

Agent Builder CLI for Kubernetes.

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model,

Behavioral testing for AI agents.

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

agy-staff, Claude Code, OpenAI Codex ve Pi kullanıcılarının Google Antigravity CLI üzerinden Gemini tabanlı bir agent’ı farklı görevler için çağırmasını sağlayan bir plugin’dir.

Neden yükseliyor · Farklı coding-agent harness’leri üzerinden agy’ye görev devretme fikri dikkat çekici; 40 günlük yaşına karşılık 652 yıldız ve son 28 günde 14 commit ile 3 farklı yazar görünür bir ilgi ve geliştirme faaliyetine işaret ediyor. Bu veriler kullanım veya kalite kanıtı değildir.

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

Evidence-gated Obsidian brain for bounded multi-agent improvement loops.

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, an

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

CatchBench: a benchmark for auditing agent failures across the full lifecycle.

The official Kotlin benchmark by JetBrains

A small, vetted, self-evolving harness for Claude Code — curated, not dumped. Skills, safety guards, and a self-improvement loop.

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

开源 AI 应用评测平台，支持 RAG、AI Agent、多轮对话、LLM-as-Judge、接口评测、评测报告和人工盲测。Open-source AI evaluation platform for RAG, AI Agents, multi-turn conversation

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

Crash-test insurance claim AI agents before production.

OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents

Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect.

Automated auditing pipeline for LLM and agent benchmarks — surfaces task ambiguity, environment conflicts, and evaluation bugs.

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

Local-first, IM-native AI coworker runtime with async role agents, trace replay, evidence, and agentic evaluation.

Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.

OR-Space: a full-lifecycle workspace benchmark for industrial optimization agents