Havuz

FailproofAI, farklı AI agent harness’lerindeki çalıştırmaları izleyen ve tehlikeli tool çağrılarını yürütülmeden önce policy ile engelleyebilen bir araçtır.

Neden yükseliyor · 175 günlük bir repo için 5.166 yıldız ve son 28 günde 84 commit ile 8 farklı yazar, hem belirgin ilgiye hem de yakın dönem geliştirme faaliyetine işaret ediyor. Çok sayıda harness için ortak observability ve enforcement sunma iddiası, agent araçlarının çeşitlendiği dönemde dikkat çekici; yıldızlar tek başına benimseme veya kalite kanıtı değil.

Waku, yerel SQLite memory, agent loop ve eval araçlarını bir araya getiren, Python tabanlı kişisel AI assistant uygulamasıdır.

Neden yükseliyor · 79 günlük bir repo için 1.849 yıldızın yanında son 28 günde 30 commit ve 7 farklı yazar bulunması görünür ilgi ve devam eden geliştirme gösteriyor. README'nin sade, okunabilir agent loop ve entegre memory/eval yaklaşımı da projeyi agent harness arayan geliştiricilere çekici kılıyor; bu metrikler benimsenme veya üretim başarısını tek başına kanıtlamıyor.

neatroots: see, check and replay your AI agent's runs on your machine while you build it. Local-first, by Neatlogs.

jevals, agent trace'lerini eval ve guardrail kontrollerinden tek istekte geçirerek production veya agent loop içinde değerlendirmeyi amaçlayan bir Python kütüphanesidir.

Neden yükseliyor · 7 günlük bir repoda 97 yıldız, 5 commit ve 4 farklı yazar erken ilgi ve katkı işareti. Tek istekte birden fazla kontrol, düşük maliyet ve agent loop'a uygun hız iddiaları dikkat çekiyor; performans ve maliyet rakamları README'de verilmiş, bağımsız doğrulama olarak kabul edilmemeli.

Joy — a local-first personal assistant. Rust core (loop / memory / tools / graph / evals), TypeScript gateways + dashboard, Python thin SDK.

Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and

Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open model

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity re

My Pi coding agent setup: secret redaction, LSP, searchable context, recall, telemetry, observability, and team mode. Toggle with /extension

Does the model pick the right tool? Tool-routing tests for your MCP server: run them on Claude and OpenAI models, see where each request lan

Developer and product skills for Claude Code, Cursor and Codex: 1-file AI evals and Chrome extension store assets.

Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

Run Inspect AI evals in the cloud

Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it te

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.

Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic sco

Decision harness for TypeSafe Jev — confidence gates, shadow mode, recipes, and evals. Claude CLI 48.9s → Jev 1.3s on the same row-filter jo

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Blind A/B testing for AI agent skills, .cursorrules, CLAUDE.md, and system prompts. Detects placebo and harmful instructions.

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit p

Benchmark LLMs head-to-head on your own prompts.

Methodology and deterministic evaluation tools for iterative improvement of skills, repositories and agent workflows, with independent revie

Phoenix-native LLM evaluation, prompt testing, model comparison, red teaming, and observability for Elixir

Eval-driven multimodal agent for identifying trees, logs, bark, leaves and wood from photographs. Dendrology is the reference domain; the e

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Cairn runs your AI and ML pipelines behind budgets, policy, approvals and an audit trail.

Claude Code plugin marketplace — agentic-engineering (spec-driven shape→decide→execute→measure→eval with adversarial review) + github-keeper

Mutation testing for your LLM evals — degrade the system under test and check whether your evals would actually catch a regression.

The canonical standard for building production-grade agentic products — autonomy × oversight, composition patterns, the 9-layer harness, a 3