Havuz

iFixAi, AI agent’ları yapılandırılabilir testler ve model judge’larıyla denetleyip puan kartı ve raporlar üreten bir Python aracıdır.

Neden yükseliyor · 153 günlük bir depoda 15.702 yıldız ve son 28 günde 9 commit ile 5 farklı yazar görünmesi güçlü ilgiye ve devam eden katkıya işaret ediyor. README’de birden fazla kullanım yolu ve agent ekosistemi entegrasyonları öne çıkarılıyor; ancak yıldızlar ürün kalitesini veya değerlendirmelerin doğruluğunu kanıtlamaz.

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

Open-source benchmark for browser AI agents on daily tasks.

Future AGI, LLM ve AI agent uygulamalarını değerlendirme, izleme, simüle etme ve koruma işlerini self-host edilebilen tek bir platformda birleştiriyor.

Neden yükseliyor · Konu, agent uygulamalarında evaluation ve observability ihtiyacının artmasıyla örtüşüyor; 157 günlük repo yaşına karşılık 2.081 yıldız ve son 28 günde 597 commit geliştirme temposuna işaret ediyor. Farklı yazar sayısı 6; bu veriler ilginin nedenini tek başına kanıtlamaz ve yıldızlar benimseme ya da kalite ölçüsü değildir.

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation,

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifac

agy-staff, Claude Code, OpenAI Codex ve Pi kullanıcılarının Google Antigravity CLI üzerinden Gemini tabanlı bir agent’ı farklı görevler için çağırmasını sağlayan bir plugin’dir.

Neden yükseliyor · Farklı coding-agent harness’leri üzerinden agy’ye görev devretme fikri dikkat çekici; 40 günlük yaşına karşılık 652 yıldız ve son 28 günde 14 commit ile 3 farklı yazar görünür bir ilgi ve geliştirme faaliyetine işaret ediyor. Bu veriler kullanım veya kalite kanıtı değildir.

An audit agent that proves whether a person's data is really gone: resolves identity across systems, judges state at the audit cutoff, finds

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Asking for What Was Never Requested: horizontal and vertical proactivity in LLM agents. Need-graph metrics (no LLM judge) and Q&D, which tra

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Agent skill for web typography: Bringhurst applied to modern CSS, with claims measured in the browser and a baseline eval suite

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark a

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/

Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, an

Behavioral testing for AI agents.

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimiz

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Declarative, local-first runtime for bounded AI agents — define an agent in one manifest, run it headless, and audit every effect.

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

Self-hosted evidence store for coding agents.

开源 AI 应用评测平台，支持 RAG、AI Agent、多轮对话、LLM-as-Judge、接口评测、评测报告和人工盲测。Open-source AI evaluation platform for RAG, AI Agents, multi-turn conversation

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.

OR-Space: a full-lifecycle workspace benchmark for industrial optimization agents

Evidence-gated Obsidian brain for bounded multi-agent improvement loops.

OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents

项目验证工作台：架构可视化、CI 检查、覆盖率分析与修复验证 | A project verification workspace for architecture maps, CI checks, coverage analysis, and evidence-backed f

Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model,

A work-centered runtime for agentic software engineering. Work persists; agents, context, and graphs assemble around it.

The official Kotlin benchmark by JetBrains

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

Production-oriented multi-agent course advisor with deterministic action governance, offline regression evaluation, SSE contracts and observ

Compact AI agent runtime control plane with governed tools, HITL approval, resumable execution, trace-driven evaluation, and SWE-bench-shape

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Verification-gated skill routing and self-improvement harness for Hermes-style agent skills

Agent trace analysis and context remediation plugin for Claude Code. Detects quality issues in your AI agent traces — goal drift, hallucinat

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.