Havuz

Mutation testing for your LLM evals — degrade the system under test and check whether your evals would actually catch a regression.

Compile an AI agent's repeated workflows into deterministic, auditable routines that replay for free, with a fallback to the agent.

Store agent traces remotely. Analyze them locally. Keep control of the data.

Run-aware token governance for multi-agent systems.

Ground truth for enterprise AI: your AI coding tools, your accounts and every session on your own server, from one web console, CLI or API.

Langfuse on GreptimeDB. Self-hosted LLM observability.

Six Claude Code skills that harden Opus 4.8 toward frontier behavior. Written by Fable 5, pressure-tested on the target model with transcrip

Operational doctrine for practical AI systems design.

Session-layer circuit breaker for LLM & agent traffic — a transparent streaming proxy that meters every session and kills the one that goes

Local proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history,

Enterprise-grade sovereign AI for organizations where SaaS AI isn’t an option.

Reference implementation of a list-building path for GTM engineers: a plain-English campaign brief becomes a deduped, qualified, enriched, s

Economical human-readable logs and structural (event pattern) retrieval for agentic trajectories

👻 MoEWatch: A lightweight diagnostic and audit library for Mixture-of-Experts (MoE) models in HuggingFace Transformers — the pytest for MoE

Prove an AI-written fix actually fixes the bug. mini-ork certify reproduces the bug against your repo's own code, attacks the patch with gen

Transparent LLM gateway trace recorder · sub2api / CLIProxyAPI / new-api 的 LLM 网关流量录制器

Deterministic cost / loop / time budgets · full observability · crash-resumable runs · human-approval gates · a memory you own. Self-hosted.

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping yo

Typed evaluation for OpenCode skills and AI agents: capability, regression and product checks; provider-neutral evidence grading, provenance

Open-source, self-hosted AI-assisted operations for Kubernetes and Linux. Start here for source, architecture, deployment, and contributions

Continuum — the agent runtime by ShyftLabs. Build, orchestrate, ship.

Bounded, inspectable LLM inference pipelines from declared YAML — runs offline against Ollama or any OpenAI-compatible local server, emittin

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

Companion code for the Manning book 'LLM Customization and Fine-Tuning.' Adapt open-weights LLMs end to end: prompting and RAG, LoRA/QLoRA,

Local outcome-loop + reward layer for coding agents — remember what worked, ship the next move, measure how it landed. Docs at nimroboai.com

End-to-end Langfuse workshop using a TypeScript Agent to teach the AI engineering loop: tracing, prompt management, monitoring, datasets, ex

A production-ready toolkit to accelerate and automate the end-to-end lifecycle of AI Agent development.

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Radicalbit AI Gateway

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Turn scattered knowledge, operational data, and history into source-linked context that your agents can inspect, explain, and reuse.

Local CLI for auditing AI agent skills, conflicts, duplicates, safety risks, and context cost across coding agents.

Audit trail database for AI agents. Tamper-evident, checksum-backed decision logs with session replay and time-travel debugging to support E

Production LLM call layer for AI agents and tools: keep OpenAI/Anthropic/AI SDK/LiteLLM, hot-swap models with MDA presets, and add cache, re

Self-hostable agentic OS. Build, run & audit multi-agent AI teams from signed plugins. Bring your own LLM key, own all your data, EU/GDPR-re

半人马环 Centaur Loop：AI 员工的最小工作单元框架。把复杂岗位拆解为可由 AI 接管、由人类治理、由反馈和记忆持续进化的循环工作流 / The smallest work unit for building AI employees.

Multi-agent orchestration & workflow engine. Declarative YAML workflows, LLM coordinator with hub-and-spoke mailboxes, race-safe delivery. O

Evidence-driven skill evolution for Hermes Agent — reports, dry-run proposals, candidate search, and guarded apply

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

Future AGI, LLM ve AI agent uygulamalarını değerlendirme, izleme, simüle etme ve koruma işlerini self-host edilebilen tek bir platformda birleştiriyor.

Neden yükseliyor · Konu, agent uygulamalarında evaluation ve observability ihtiyacının artmasıyla örtüşüyor; 157 günlük repo yaşına karşılık 2.081 yıldız ve son 28 günde 597 commit geliştirme temposuna işaret ediyor. Farklı yazar sayısı 6; bu veriler ilginin nedenini tek başına kanıtlamaz ve yıldızlar benimseme ya da kalite ölçüsü değildir.

Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routin

The operations layer for your local LLM stack

LLM spend tracking and budgets for Rails — by user, feature, or any tag, in your own database, no proxy.

Verdict-first output for AI coding agents. Tiny prompt + installer for Claude Code, Codex, Gemini, Cursor, opencode, and 30+ agents.

Bring your own key (BYOK) relay to enable lightweight frontend-only apps to use LLMs using the user's API keys without hitting CORS.

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — de

Run coding agents without babysitting them. Keep jobs focused, bounded, checked and accountable from start to finish. Finally run your agent

Claw is a local-first AI control plane that runs powerful open models on your machine and connects to top LLM providers. It intelligently ro

Open-source AI gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking.

An open source, Native AI Gateway and LLM proxy built in Rust