Havuz

iFixAi, AI agent’ları yapılandırılabilir testler ve model judge’larıyla denetleyip puan kartı ve raporlar üreten bir Python aracıdır.

Neden yükseliyor · 153 günlük bir depoda 15.702 yıldız ve son 28 günde 9 commit ile 5 farklı yazar görünmesi güçlü ilgiye ve devam eden katkıya işaret ediyor. README’de birden fazla kullanım yolu ve agent ekosistemi entegrasyonları öne çıkarılıyor; ancak yıldızlar ürün kalitesini veya değerlendirmelerin doğruluğunu kanıtlamaz.

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs tha

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifac

Open-source benchmark for browser AI agents on daily tasks.

Future AGI, LLM ve AI agent uygulamalarını değerlendirme, izleme, simüle etme ve koruma işlerini self-host edilebilen tek bir platformda birleştiriyor.

Neden yükseliyor · Konu, agent uygulamalarında evaluation ve observability ihtiyacının artmasıyla örtüşüyor; 157 günlük repo yaşına karşılık 2.081 yıldız ve son 28 günde 597 commit geliştirme temposuna işaret ediyor. Farklı yazar sayısı 6; bu veriler ilginin nedenini tek başına kanıtlamaz ve yıldızlar benimseme ya da kalite ölçüsü değildir.

agy-staff, Claude Code, OpenAI Codex ve Pi kullanıcılarının Google Antigravity CLI üzerinden Gemini tabanlı bir agent’ı farklı görevler için çağırmasını sağlayan bir plugin’dir.

Neden yükseliyor · Farklı coding-agent harness’leri üzerinden agy’ye görev devretme fikri dikkat çekici; 40 günlük yaşına karşılık 652 yıldız ve son 28 günde 14 commit ile 3 farklı yazar görünür bir ilgi ve geliştirme faaliyetine işaret ediyor. Bu veriler kullanım veya kalite kanıtı değildir.

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

An audit agent that proves whether a person's data is really gone: resolves identity across systems, judges state at the audit cutoff, finds

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Agent skill for web typography: Bringhurst applied to modern CSS, with claims measured in the browser and a baseline eval suite

An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark a

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.

Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which tra

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

EC2-Bench (EC²-Bench): benchmark for edge-cloud collaboration in long-horizon LLM agents — task performance, resource usage & privacy exposu

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/

Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

开源 AI 应用评测平台，支持 RAG、AI Agent、多轮对话、LLM-as-Judge、接口评测、评测报告和人工盲测。Open-source AI evaluation platform for RAG, AI Agents, multi-turn conversation

A work-centered runtime for agentic software engineering. Work persists; agents, context, and graphs assemble around it.

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, an

The official Kotlin benchmark by JetBrains

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Real-world browser-agent benchmark: 210 tasks across 107 websites, multi-agent/multi-browser evaluation, reproducible leaderboard and result

Self-hosted evidence store for coding agents.

Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.

Evidence-gated Obsidian brain for bounded multi-agent improvement loops.

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model,

Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.

Production-oriented multi-agent course advisor with deterministic action governance, offline regression evaluation, SSE contracts and observ

Compact AI agent runtime control plane with governed tools, HITL approval, resumable execution, trace-driven evaluation, and SWE-bench-shape

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Agent trace analysis and context remediation plugin for Claude Code. Detects quality issues in your AI agent traces — goal drift, hallucinat

Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect.

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors,