Havuz

iFixAi, AI agent’ları yapılandırılabilir testler ve model judge’larıyla denetleyip puan kartı ve raporlar üreten bir Python aracıdır.

Neden yükseliyor · 153 günlük bir depoda 15.702 yıldız ve son 28 günde 9 commit ile 5 farklı yazar görünmesi güçlü ilgiye ve devam eden katkıya işaret ediyor. README’de birden fazla kullanım yolu ve agent ekosistemi entegrasyonları öne çıkarılıyor; ancak yıldızlar ürün kalitesini veya değerlendirmelerin doğruluğunu kanıtlamaz.

Future AGI, LLM ve AI agent uygulamalarını değerlendirme, izleme, simüle etme ve koruma işlerini self-host edilebilen tek bir platformda birleştiriyor.

Neden yükseliyor · Konu, agent uygulamalarında evaluation ve observability ihtiyacının artmasıyla örtüşüyor; 157 günlük repo yaşına karşılık 2.081 yıldız ve son 28 günde 597 commit geliştirme temposuna işaret ediyor. Farklı yazar sayısı 6; bu veriler ilginin nedenini tek başına kanıtlamaz ve yıldızlar benimseme ya da kalite ölçüsü değildir.

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs tha

Open-source benchmark for browser AI agents on daily tasks.

agy-staff, Claude Code, OpenAI Codex ve Pi kullanıcılarının Google Antigravity CLI üzerinden Gemini tabanlı bir agent’ı farklı görevler için çağırmasını sağlayan bir plugin’dir.

Neden yükseliyor · Farklı coding-agent harness’leri üzerinden agy’ye görev devretme fikri dikkat çekici; 40 günlük yaşına karşılık 652 yıldız ve son 28 günde 14 commit ile 3 farklı yazar görünür bir ilgi ve geliştirme faaliyetine işaret ediyor. Bu veriler kullanım veya kalite kanıtı değildir.

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifac

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model,

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

A work-centered runtime for agentic software engineering. Work persists; agents, context, and graphs assemble around it.

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, an

Behavioral testing for AI agents.

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.

The official Kotlin benchmark by JetBrains

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

Production-oriented multi-agent course advisor with deterministic action governance, offline regression evaluation, SSE contracts and observ

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

Compact AI agent runtime control plane with governed tools, HITL approval, resumable execution, trace-driven evaluation, and SWE-bench-shape

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark a

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Agent trace analysis and context remediation plugin for Claude Code. Detects quality issues in your AI agent traces — goal drift, hallucinat

Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect.

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.

An audit agent that proves whether a person's data is really gone: resolves identity across systems, judges state at the audit cutoff, finds

Crash-test insurance claim AI agents before production.

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

Self-hosted evidence store for coding agents.

Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors,

Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which tra

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Automated auditing pipeline for LLM and agent benchmarks — surfaces task ambiguity, environment conflicts, and evaluation bugs.