Havuz

🤖 LLM Engineering Roadmap — Complete Developer Guide

AI 执行与监察体系（Agent Oversight Framework）：分层治理规范 + L2 运行时监察参照实现——证据分级、自校正、边界诚实。A layered governance spec for AI agents, plus a runnable referenc

sofagent is an open-source FDE Harness for AI coding agents — on entry, write your business judgment into files (workflows, ontology, AI nod

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

Chinese-first LLM jailbreak benchmark & red-team toolkit | 中文LLM安全评测框架 | by POJIA.AI

A modular threat detection library for AI and LLM pipelines. It provides language-agnostic guardrails through a configurable pipeline of loa

Open-source AI model evaluation and benchmarking framework for LLMs (OpenAI, Ollama, Claude, Gemini)

A Socratic supervision layer for AI coding agents.

AI Firewall & LLM security toolkit - protect your AI applications from prompt injection, jailbreaks, PII leakage, and adversarial attacks

End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer, render heatmaps and cooking cur

Every way an LLM can break one SDK to stop it. Injection. Hallucination. Leakage. Handled.

Open database of system prompts extracted from frontier LLMs using JustAsk

Biologically-grounded adversarial training platform: cyclic Wake/Dream/Nightmare/Compress phases that accumulate model robustness without ca

🎱 AI Agent Governance Framework — Constrain how AI Agents behave in your project. pip install pattern8

High-fidelity Claude Fable 5 (Mythos) environment emulation and automated multi-agent jailbreak (Pack Hunt) research laboratory.

Verify what AI coding agents can actually read, write, execute and reach using synthetic canaries.

Behavioral testing for AI agents.

A custom coding harness for TypeSafe AI's Jev: an LLM proposes, Jev answers narrow questions, code decides, every step leaves a receipt.

Delta MCP is a free app that sits between your AI apps and their MCP servers: any server, every tool. It checks each task before it runs, ap

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Reproducible OpenEnv-compatible reinforcement-learning environments for social-media integrity moderation.

Detect hallucinations in LLM responses. Verify every claim against source documents using hybrid STS + NLI. Works with LangChain, LlamaIndex

Myrtia — agent-native red teaming for LLMs and AI agents: a deliberating council of attacker models (propose, cross-critique, Borda vote, ch

Overnight experiment: can an AI agent make money, and can you trust what it reports? The agent produces claims; a separate verifier it canno

Free local guardrail for AI coding agents: every tool_call judged on-device (regex + Laya), allow/review/block

1,017 system prompts from real AI products, audited span by span against the eight AISPA assurance dimensions

A terminal-based, multi-domain support-ticket triage agent grounded in a local support corpus.

Tamper-evident evidence and verification layer for AI agents. Verify any agent's session offline with just openssl. No cloud, no lock-in.

Lightweight RAG provenance middleware. Verifies every claim in an LLM response is grounded in a retrieved source - without an LLM call.

Zero-cost epistemic uncertainty quantification & hallucination detection for LLMs (90,000x faster than Semantic Entropy)

AI makes users productive — but also cognitively lazy. Lucid detects cognitive dependency in real-time and adapts AI responses to keep users

⚡ Millisecond System-1 judgement for every tool call in DeepSeek Harness — Jev-powered risk classification & evidence-gated auto-approval. F

The gate between an AI agent and real money. Runtime authorization for financial AI agents: structured intent, deterministic policy, human a

AI agents now operate with authority. Authority without discipline is how complex systems fail. Nuclear’s control loop, ported to AI-assiste

A structured workflow system for AI coding agents - harness engineering, execution loop, skills, hooks, and a learning loop. Works with Clau

🔥 CogitoAgent — An autonomous AI agent, cloud-driven but locally executed. It continuously thinks, automatically analyzes, and integrates pr

MCTS (Model Context Threat Scanner) is a local-first security scanner for MCP servers -- static and live tool discovery, multiple analyzers,

RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)

Audit any agent decision across its past, present, and future, on one typed graph.

Local-first verification layer for AI agents: evidence, provenance, policy gates, human approval, and auditable Trust Receipts before a clai

An open database of real-world AI agent incidents.

Checks every AI agent action against your company's real rules before it runs. Git-native, PR-reviewed.

Noisegate: a differential privacy gateway that lets an untrusted LLM agent query sensitive data over MCP (Model Context Protocol), with a fo

Python SDK for LLM guardrails with safety classification, PII detection, prompt injection defense, and grounding checks. Protect AI applicat

Parent-operated starter kit for designing a private, child-facing Hermes Agent profile

Prompt level, inference time, AI frameworks to improve safety and alignment.

The execution safety layer for AI agents.

Runtime security platform for LLM agent tool calls on Kubernetes

Declare the outcome, skip the logic. The developer-first infrastructure engine for structured, multi-turn AI conversations with built-in saf

Methodology and deterministic evaluation tools for iterative improvement of skills, repositories and agent workflows, with independent revie