Havuz

A benchmark for testing whether coding agents can resolve engineering tasks in scientific software

rook by TestMu AI: Agent Assurance from the terminal. Test how your AI agents actually behave across workflows, tools, and actions, and catc

Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario

Mechanical Governance for LLM Decisions — model-agnostic governance regimes (R1/R2/R3), hard gates, entropy commit-reveal and governance met

Paper artifacts, figures, aggregate data, and share cards for the DXRG paper: What LLM Trading Agents Actually Do in Production (six months,

Live trading arena for LLM agents: public reasoning, real market data, virtual capital

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

discover LLMs punching above their weight

End-to-end Langfuse workshop using a TypeScript Agent to teach the AI engineering loop: tracing, prompt management, monitoring, datasets, ex

Multi-Provider AI Gateway, "Because we have LiteLLM at home" No personal logs by design. Model autodiscovery, Failover groups, High availabi

Hybrid rule-based and context-aware LLM evaluation system for Chinese-to-Japanese game localization QA.

Assessment-driven, local-first learning agent built on an observable Agent Runtime and eval harness—grounded ingestion, trace replay, HITL a

多智能体小说写作框架 | Multi-agent novel-writing framework — 每个角色都有自己的脑子,每一章都有质量门禁

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results

PaperBench High-Difficulty and Expanded Tasks: paper reproduction beyond machine learning. 93 tasks in 12 research areas, run in domain-nati

Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic sco

When asked to pick a random number between 1 and 100, ChatGPT does not follow a random uniform distribution

A design-of-experiments platform for evaluating compound AI systems - find which technique drives quality, by how much, and whether the di

Phoenix-native LLM evaluation, prompt testing, model comparison, red teaming, and observability for Elixir

Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

Lightweight RAG provenance middleware. Verifies every claim in an LLM response is grounded in a retrieved source - without an LLM call.

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Combine the results from a panel of models into an enhanced response

DiagSafe: diagnostic LLM safety benchmark with prompt-level rubrics (3,489 bilingual prompts, 84-label taxonomy)

Myrtia — agent-native red teaming for LLMs and AI agents: a deliberating council of attacker models (propose, cross-critique, Borda vote, ch

CI for your coding-agent setup (CLAUDE.md, skills, hooks): pinned baselines, a canary on every Claude Code release, model and version matrix

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

A pure-Python structured memory benchmark for multi-agent LLM systems — context graph vs vector RAG vs raw history dump, five scenarios, 18

RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)

Runtime security for AI agents. In-process, zero dependencies, Apache 2.0.

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

pytest for AI Apps — Fast, free, offline testing and self-improving prompts for Python developers.

A production GenAI architecture guide for Forward Deployed Engineer (FDE) and Applied AI interviews—RAG, agents, evals, security, and operat

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

A preservation-aware benchmark for natural-language SVG repair with deterministic binary rewards and complete model traces.

Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it te

Reference harness for the Big Finance benchmark of workflow-grounded financial-research questions

Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and

Behavioral regression testing for AI apps and agents: pytest plugin, behavior diff, CI gate, and document extraction metrics

AIRadar · 每日定时运行的 AI 行业情报工作流

Your Evals Are Lying To You: a hands-on TypeScript curriculum for LLM evals, from a five-line assertion to a live diagnostic console. By Tej

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit p

AI 模拟面试官：每条扣分都强制引用你回答的逐字原文（机器校验，编造引用会被重试/丢弃），四维度加权由确定性代码计算而非模型现编，评分体系设计为可经真人盲评校准 · FastAPI + Vue3 + LangGraph

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Diagnosing policy grounding in LLM judges via causal erasure. Companion code to the AIES 2026 submission READ.

Developer and product skills for Claude Code, Cursor and Codex: 1-file AI evals and Chrome extension store assets.

甲作 Atelier：手机端 AI 美甲推荐与虚拟试戴；仓库公开其中的轻量评测工作台。