Havuz

Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate,

Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic sco

CoDA-Bench is a benchmark for code agents on data-intensive tasks. 🎈代码智能体能搞定数据密集型任务吗?

GameOps Investigator, oyun analitiğindeki düşüş ve anomali vakalarını read-only SQL, istatistiksel karşılaştırmalar ve incelemeye açık raporlarla araştıran bir araçtır.

Neden yükseliyor · 76 yıldız ve 28 günde 23 commit ilgi gördüğünü gösteriyor; oyun analitiğini agent ve MCP yaklaşımıyla birleştirmesi dikkat çekmiş olabilir. Yükselişin kesin nedenini gösteren veri yok; commit’lerin tek yazardan gelmesi topluluk katkısının henüz sınırlı olduğuna işaret ediyor.

Phoenix-native LLM evaluation, prompt testing, model comparison, red teaming, and observability for Elixir

Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

Jev-powered, rule-based grader for text files. Runs every rule against every line in parallel. No skimming, no missed lines.

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Lightweight RAG provenance middleware. Verifies every claim in an LLM response is grounded in a retrieved source - without an LLM call.

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Combine the results from a panel of models into an enhanced response

jev-calibrate, Jev sorularının etiketli örneklerde karar vermeye uygunluğunu ölçen bir TypeScript CLI aracıdır.

Neden yükseliyor · Kesin yükseliş nedeni mevcut verilerden anlaşılamıyor. Repo 6 günlük, 32 yıldızlı ve son 28 günde 11 commit almış; ancak farklı yazar sayısının 1 olması aktivitenin tek geliştiricide toplandığını gösteriyor.

A pure-Python structured memory benchmark for multi-agent LLM systems — context graph vs vector RAG vs raw history dump, five scenarios, 18

RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)

Runtime security for AI agents. In-process, zero dependencies, Apache 2.0.

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

A production GenAI architecture guide for Forward Deployed Engineer (FDE) and Applied AI interviews—RAG, agents, evals, security, and operat

pytest for AI Apps — Fast, free, offline testing and self-improving prompts for Python developers.

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

A preservation-aware benchmark for natural-language SVG repair with deterministic binary rewards and complete model traces.

Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it te

AIRadar · 每日定时运行的 AI 行业情报工作流

Your Evals Are Lying To You: a hands-on TypeScript curriculum for LLM evals, from a five-line assertion to a live diagnostic console. By Tej

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit p

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Diagnosing policy grounding in LLM judges via causal erasure. Companion code to the AIES 2026 submission READ.

甲作 Atelier：手机端 AI 美甲推荐与虚拟试戴；仓库公开其中的轻量评测工作台。

面向小团队的一站式 NLP 小模型平台:数据集版本管理 · 训练 · 评估 · 在线部署 · Badcase 闭环,内置大模型 Prompt 盲测评测(人工 + AI 双指标)。

Open-source agent reliability platform for discovering, investigating, and fixing production agent failures

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors,

Same-brief agentic build comparison: GPT-6 Astra High vs Gemini 3.8 Flash High building a playable 3D AI civilization simulation game.

Adversarial LLM Reversal for hermes-agent

My complete journey to becoming an Agentic AI Engineer through structured learning, projects, experiments, and production-ready implementati

Snapshot testing for LLM apps and agents, built to run locally and block regressions in CI.

Regression testing and configuration sweeps for RAG pipelines, with the statistics to know whether a change actually helped.

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

Typed decisions (choice / score / yes-no) from local Qwen models on Apple Silicon. Probabilities come straight from the logits, no text gene

The Regression Testing Framework for AI Agents. Replay · Evaluate · Assert · Catch Regressions — in CI. Like Jest for your AI layer.