Havuz

Phoenix-native LLM evaluation, prompt testing, model comparison, red teaming, and observability for Elixir

Langfuse on GreptimeDB. Self-hosted LLM observability.

Economical human-readable logs and structural (event pattern) retrieval for agentic trajectories

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

A 21-day hands-on journey to mastering AI for DevOps — covering LLMs, GenAI, CI/CD, Kubernetes, cloud, automation, and AI-powered DevOps pro

Aixle Flow — orchestrate coding agents through durable, inspectable workflows.

Myrtia — agent-native red teaming for LLMs and AI agents: a deliberating council of attacker models (propose, cross-critique, Borda vote, ch

Self-hostable agentic OS. Build, run & audit multi-agent AI teams from signed plugins. Bring your own LLM key, own all your data, EU/GDPR-re

Prove an AI-written fix actually fixes the bug. mini-ork certify reproduces the bug against your repo's own code, attacks the patch with gen

Audit any agent decision across its past, present, and future, on one typed graph.

Python SDK for LLM guardrails with safety classification, PII detection, prompt injection defense, and grounding checks. Protect AI applicat

Compile an AI agent's repeated workflows into deterministic, auditable routines that replay for free, with a fallback to the agent.

端到端智能体平台：把本机的 AI Agent 变成人人可用的 Web 产品（Vue 3 + Flask + Go WS 网关 + Docker 部署）

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decodi

The operations layer for your local LLM stack

Open-source LLM control plane: observe every call, recommend savings, then apply only human-approved, reversible routing.

200+ AI, Gen AI, LLM, agentic AI, Forward Deployed Engineer (FDE), AI systems architect, applied AI, LLMOps and AI platform engineering inte

Observable diagnostics, failure attribution, metrics, and no-key replay for multi-agent runtimes.

Enterprise-ready CI/CD reference for Microsoft Foundry AI agents, with parallel GitHub Actions and Azure DevOps pipelines, evaluation-driven

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.

Proxima lets existing GPUs serve 4x more concurrent requests

CONTINUUM: Verifiable semantic recovery for long-running AI agents. Semantic checkpoints (not conversation dumps), an idempotent action ledg

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → paralleli

Benchmark LLMs head-to-head on your own prompts.

Claw is a local-first AI control plane that runs powerful open models on your machine and connects to top LLM providers. It intelligently ro

Open-source control plane for private, governed enterprise AI — one gateway for every model, composable governed pipelines, and apps your wh

Behavioral regression testing for AI apps and agents: pytest plugin, behavior diff, CI gate, and document extraction metrics

Verdict-first output for AI coding agents. Tiny prompt + installer for Claude Code, Codex, Gemini, Cursor, opencode, and 30+ agents.

A token-spend profiler and cost-regression gate for AI agents.

Declarative Python SDK for building AI inference platforms on Kubernetes — bare machines to a served model, composed rather than hand-assemb

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit p

Compression with a quality contract — cache-aware, causally-pruned LLM context compression for agentic runtimes, certified non-inferior acro

Enterprise-grade sovereign AI for organizations where SaaS AI isn’t an option.

A production-ready toolkit to accelerate and automate the end-to-end lifecycle of AI Agent development.

Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model

Deterministic cost / loop / time budgets · full observability · crash-resumable runs · human-approval gates · a memory you own. Self-hosted.

Local, self-hosted agent infrastructure — one endpoint for every AI client (Claude Code, Codex CLI, Gemini CLI, OpenCode). Unified model acc

Companion code for the Manning book 'LLM Customization and Fine-Tuning.' Adapt open-weights LLMs end to end: prompting and RAG, LoRA/QLoRA,

Ground truth for enterprise AI: your AI coding tools, your accounts and every session on your own server, from one web console, CLI or API.

Control plane for AI coding agents: route tasks, reduce token spend, run multi-agent workflows, fallback executors, and track cost per task.

Gates 84.7% of tool-schema prompt tokens away for DeepSeek Harness: plugins declare themselves organs, a nerve impulse routes each command,

Open-source, self-hosted AI-assisted operations for Kubernetes and Linux. Start here for source, architecture, deployment, and contributions

Reference implementation of a list-building path for GTM engineers: a plain-English campaign brief becomes a deduped, qualified, enriched, s

Session-layer circuit breaker for LLM & agent traffic — a transparent streaming proxy that meters every session and kills the one that goes

👻 MoEWatch: A lightweight diagnostic and audit library for Mixture-of-Experts (MoE) models in HuggingFace Transformers — the pytest for MoE

Snapshot testing for LLM apps and agents, built to run locally and block regressions in CI.

Regression testing and configuration sweeps for RAG pipelines, with the statistics to know whether a change actually helped.

The whole eve stack. On your own machine. Durable Postgres sessions, a Docker sandbox, memory, approvals, schedules, 1,070 tool integrations

Commented SHIP? Full briefs for 3 pile-two AI engineering projects: LLM Ops, Agent Evals, Auto Routing. What to build, what to measure, what

Local CLI for auditing AI agent skills, conflicts, duplicates, safety risks, and context cost across coding agents.