Havuz

Local-first AI coding agent runtime — your models, fail-closed safety, evidence-gated self-improvement

An autonomous agentic pipeline that finds, proves, and patches real C memory-safety vulnerabilities end-to-end using a single 7B open-source

Diagnose vLLM inference servers

AMD Radeon 9700 AI PRO RDNA multi GPU Tools for llm/vLLM inference, tuning, overclocking, undervolting and LLM models benchmarking

Live per-node GPU, host, and vLLM stats for DGX Spark clusters, in your terminal

Open-source local AI server configs, GFX906 runtime maintenance, reproducible benchmarks, and QC methods for affordable AI research infrastr

Honeycomb Lab — hex map + OpenAI gateway control plane for a home AI fleet

Instinct: single-forward-pass decision models (choice / true-false / score) returning a probability per candidate

A dashboard for a home cluster of NVIDIA GB10 (DGX Spark / ASUS GX10) inference servers: GPU and system health, plus live per-model metrics

Composable LLM decoding on the probability simplex: library and paper code.

Thinking-native compiler for LEET Language Understanding item sets — killer-item design grammar, verification separation, deterministic gate

Hikari's curated English measured notes on local LLM systems (serving, specdec, training, steering).

让 MinerU 在 AMD 显卡上以 vLLM + hybrid-auto-engine 满血运行！ 本项目是对官方文档中 AMD/ROCm 生态支持的有效补充。针对 MinerU 3.x 提供了一套完整的 ROCm 7.x + PyTorch 2.11.0 + vLLM 源码

The Continuous Verification and Optimization layer for self-hosted LLMs

Did your change make your LLM server faster? Measure it: faster, slower, or too close to call. Linux and Apple Silicon.

Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks

Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with h

Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the be

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

Think only when the tool call looks unsure: confidence-gated reasoning for small local models. Code, BFCL results and paper.

OpenAI-compatible multilingual TTS server — Chatterbox on vLLM with real-time PCM audio streaming, low time-to-first-byte (~0.7 s), voice cl

模型路由器：在 Codex Desktop 和 Claude Desktop 使用 Claude、第三方 API 与自定义模型。Local LLM proxy for Codex & Claude Desktop: OpenAI Responses / Chat Completi

Serve DeepSeek and Qwen, run an on-demand model library, and fine-tune with Unsloth QLoRA on 2x NVIDIA DGX Spark. TP=2 vLLM lanes behind one