Havuz

Self-hosted system for improving AI agents, by InferEdge

Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and

Does the model pick the right tool? Tool-routing tests for your MCP server: run them on Claude and OpenAI models, see where each request lan

Benchmark of AI coding agents building web apps and deploying them to Cloudish, DigitalOcean, Render and Vercel, measuring success, time, to

neatroots: see, check and replay your AI agent's runs on your machine while you build it. Local-first, by Neatlogs.

Developer and product skills for Claude Code, Cursor and Codex: 1-file AI evals and Chrome extension store assets.

What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Claude Code/Codex skills, evals, harnesses/context tools, and learning resources that make it easy to build with Jev, TypeSafe's System One

jev-calibrate, Jev sorularının etiketli örneklerde karar vermeye uygunluğunu ölçen bir TypeScript CLI aracıdır.

Neden yükseliyor · Kesin yükseliş nedeni mevcut verilerden anlaşılamıyor. Repo 6 günlük, 32 yıldızlı ve son 28 günde 11 commit almış; ancak farklı yazar sayısının 1 olması aktivitenin tek geliştiricide toplandığını gösteriyor.

jevals, agent trace'lerini eval ve guardrail kontrollerinden tek istekte geçirerek production veya agent loop içinde değerlendirmeyi amaçlayan bir Python kütüphanesidir.

Neden yükseliyor · 7 günlük bir repoda 97 yıldız, 5 commit ve 4 farklı yazar erken ilgi ve katkı işareti. Tek istekte birden fazla kontrol, düşük maliyet ve agent loop'a uygun hız iddiaları dikkat çekiyor; performans ve maliyet rakamları README'de verilmiş, bağımsız doğrulama olarak kabul edilmemeli.

Joy — a local-first personal assistant. Rust core (loop / memory / tools / graph / evals), TypeScript gateways + dashboard, Python thin SDK.

Fast, typed Python framework for Jev-as-a-Judge/Decision-model evaluation for LLMs, RAG, and AI agents along with human-label calibration.

Decision harness for TypeSafe Jev — confidence gates, shadow mode, recipes, and evals. Claude CLI 48.9s → Jev 1.3s on the same row-filter jo

Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it te

Transcript analysis for long-horizon agent evaluations

Personal AI agent on Cloudflare Workers: Durable Object per conversation, markdown memory in R2, vector recall in SQLite, fan-out deep resea

CI for your coding-agent setup (CLAUDE.md, skills, hooks): pinned baselines, a canary on every Claude Code release, model and version matrix

Research-backed, eval-driven skills for AI agents

A composable framework for reproducible evaluation of long-running agents.

Type-aware, evidence-backed delivery contract for AI coding harnesses. Different agents in. One predictable human handoff out.

Eval-driven multimodal agent for identifying trees, logs, bark, leaves and wood from photographs. Dendrology is the reference domain; the e

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit p

Agent skills for the Agentic SDLC: team lifecycle (team-boot, team-learn, team-init, team-repair), software factory, evals, CDR lifecycle wi

Open-source control plane for governed ModelOps — fine-tune, evaluate, promote, and serve models as cost-capped, policy-gated, audited comma

Your Evals Are Lying To You: a hands-on TypeScript curriculum for LLM evals, from a five-line assertion to a live diagnostic console. By Tej

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

A coding agent that builds it's own tools and mines it's own skills and always ships with receipts.

Waku, yerel SQLite memory, agent loop ve eval araçlarını bir araya getiren, Python tabanlı kişisel AI assistant uygulamasıdır.

Neden yükseliyor · 79 günlük bir repo için 1.849 yıldızın yanında son 28 günde 30 commit ve 7 farklı yazar bulunması görünür ilgi ve devam eden geliştirme gösteriyor. README'nin sade, okunabilir agent loop ve entegre memory/eval yaklaşımı da projeyi agent harness arayan geliştiricilere çekici kılıyor; bu metrikler benimsenme veya üretim başarısını tek başına kanıtlamıyor.

Skills for your skills. Create, test, audit, package, and visualize Agent Skill libraries—locally.

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity re

Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic sco

Make Claude Opus 4.8 behave like Claude Fable 5 — doctrine output style, drift-catching hooks, and an eval loop against golden Fable transcr

Benchmark LLMs head-to-head on your own prompts.

Claude Code plugin marketplace — agentic-engineering (spec-driven shape→decide→execute→measure→eval with adversarial review) + github-keeper

Methodology and deterministic evaluation tools for iterative improvement of skills, repositories and agent workflows, with independent revie

Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open model

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

A langGraph-powered multi-agent RAG assistant for warranty, return, and consumer decision workflow.

Blind A/B testing for AI agent skills, .cursorrules, CLAUDE.md, and system prompts. Detects placebo and harmful instructions.

How do you KNOW if a 'skill' is any good - deterministically? This runs controlled skill-vs-no-skill experiments. It doesn't just ask, “Did

The canonical standard for building production-grade agentic products — autonomy × oversight, composition patterns, the 9-layer harness, a 3

Canonical memory infrastructure: BM25 recall over trusted files for AI agents.

Open-source skills for Claude Code and Codex, with an MCP server, a PyPI package and Suede Awesome Everything, a directory that scores AI ag

Run your real agent with and without your skills, MCPs, and rules. See which ones actually help, and what they cost in tokens. Supports Clau

Behavioral eval lab (Quorum) for the superpowers project that drives real coding-agent CLIs (Claude, Codex, Gemini, Kimi, and more) through

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Two Claude Code skills for adversarial code review (single-model PAR and multi-model MMAR with cross-critique to catch hallucinations), plus