Havuz

Local-first CLI for benchmarking LLMs on real hardware — quality, speed, reliability, and a real multi-turn agent loop.

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

Lightweight proxy for LLM

Open, calibrated, single-pass typed-decision models (4B & 27B) for finance and trading

Self-hosted, distributed, multi-model LLM inference

Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simula

croit AIplane — one plane for all your AI. Self-hosted AI infrastructure connecting applications and users with models, agents, tools and en

A web-based memory usage and performance calculator for Huggingface GGUF models

Local-first desktop assistant for the whole job hunt — scrape listings, match with a local LLM, generate applications in your own voice, and

AI/LLM on Kubernetes 基础设施知识库：涵盖 GPU、大数据、调度、分布式训练、LLM 推理、RAG、Agent 与生产运维。| AI/LLM on Kubernetes Infrastructure Knowledge Base: Covering GPUs,

Consumer-aware compression for embedding indexes and LLM KV caches — compress by the metric the downstream consumer actually uses. PCA-Matry

Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audio ·

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

Run the LLM Swarm Router on machines to distribute Local Ai to the Swarm - More Machines - MORE SPEED

Proxima lets existing GPUs serve 4x more concurrent requests

vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → paralleli

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a lapt

MLOps — vLLM + LiteLLM + Portainer CE + Netdata. Case study song ngữ: kiến trúc, metrics, bài học vận hành. · Self-hosted LLM inference don

Tenstorrent backend plugin for vLLM.

nvtop for vLLM — an interactive terminal dashboard for vLLM serving performance (concurrency, throughput, cache & KV memory, latency, spec-d

Local-first AI coding agent runtime — your models, fail-closed safety, evidence-gated self-improvement

An autonomous agentic pipeline that finds, proves, and patches real C memory-safety vulnerabilities end-to-end using a single 7B open-source

Open-source local AI server configs, GFX906 runtime maintenance, reproducible benchmarks, and QC methods for affordable AI research infrastr

Honeycomb Lab — hex map + OpenAI gateway control plane for a home AI fleet

Thinking-native compiler for LEET Language Understanding item sets — killer-item design grammar, verification separation, deterministic gate

Hikari's curated English measured notes on local LLM systems (serving, specdec, training, steering).

The Continuous Verification and Optimization layer for self-hosted LLMs

Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks

Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the be

OpenAI-compatible multilingual TTS server — Chatterbox on vLLM with real-time PCM audio streaming, low time-to-first-byte (~0.7 s), voice cl

Serve DeepSeek and Qwen, run an on-demand model library, and fine-tune with Unsloth QLoRA on 2x NVIDIA DGX Spark. TP=2 vLLM lanes behind one