Havuz

iFixAi, AI agent’ları yapılandırılabilir testler ve model judge’larıyla denetleyip puan kartı ve raporlar üreten bir Python aracıdır.

Neden yükseliyor · 153 günlük bir depoda 15.702 yıldız ve son 28 günde 9 commit ile 5 farklı yazar görünmesi güçlü ilgiye ve devam eden katkıya işaret ediyor. README’de birden fazla kullanım yolu ve agent ekosistemi entegrasyonları öne çıkarılıyor; ancak yıldızlar ürün kalitesini veya değerlendirmelerin doğruluğunu kanıtlamaz.

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

Open-source benchmark for browser AI agents on daily tasks.

Future AGI, LLM ve AI agent uygulamalarını değerlendirme, izleme, simüle etme ve koruma işlerini self-host edilebilen tek bir platformda birleştiriyor.

Neden yükseliyor · Konu, agent uygulamalarında evaluation ve observability ihtiyacının artmasıyla örtüşüyor; 157 günlük repo yaşına karşılık 2.081 yıldız ve son 28 günde 597 commit geliştirme temposuna işaret ediyor. Farklı yazar sayısı 6; bu veriler ilginin nedenini tek başına kanıtlamaz ve yıldızlar benimseme ya da kalite ölçüsü değildir.

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifac

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs tha

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

agy-staff, Claude Code, OpenAI Codex ve Pi kullanıcılarının Google Antigravity CLI üzerinden Gemini tabanlı bir agent’ı farklı görevler için çağırmasını sağlayan bir plugin’dir.

Neden yükseliyor · Farklı coding-agent harness’leri üzerinden agy’ye görev devretme fikri dikkat çekici; 40 günlük yaşına karşılık 652 yıldız ve son 28 günde 14 commit ile 3 farklı yazar görünür bir ilgi ve geliştirme faaliyetine işaret ediyor. Bu veriler kullanım veya kalite kanıtı değildir.

An audit agent that proves whether a person's data is really gone: resolves identity across systems, judges state at the audit cutoff, finds

Agent skill for web typography: Bringhurst applied to modern CSS, with claims measured in the browser and a baseline eval suite

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which tra

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

Behavioral testing for AI agents.

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark a

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

Self-hosted evidence store for coding agents.

Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact

开源 AI 应用评测平台，支持 RAG、AI Agent、多轮对话、LLM-as-Judge、接口评测、评测报告和人工盲测。Open-source AI evaluation platform for RAG, AI Agents, multi-turn conversation

Evidence-gated Obsidian brain for bounded multi-agent improvement loops.

OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model,

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

A work-centered runtime for agentic software engineering. Work persists; agents, context, and graphs assemble around it.

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, an

Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.

The official Kotlin benchmark by JetBrains

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

Production-oriented multi-agent course advisor with deterministic action governance, offline regression evaluation, SSE contracts and observ

Compact AI agent runtime control plane with governed tools, HITL approval, resumable execution, trace-driven evaluation, and SWE-bench-shape

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Agent trace analysis and context remediation plugin for Claude Code. Detects quality issues in your AI agent traces — goal drift, hallucinat

Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect.

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.

Crash-test insurance claim AI agents before production.