Havuz

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how

Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVID

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decodi

Self-hosted homelab dashboard in one container — GPU, local-AI VRAM, Docker, systemd, energy costs. Read-only MCP server so AI agents can ex

Qwen3.8-Flash-Next (180B MoE) on one 12-16 GB NVIDIA GPU, Linux or Windows. Own engine, own compression.

Local-first AI workbench for OCR, transcription, translation, image generation, mind maps and composable workflows. Built for DGX Spark.

Windows utility focused on cuda setup, nvidia local AI, and cuda environment for local AI workflows.

Offline-first Python AI agent that runs a tiny research business: quotes each job against its own costs, collects via Stripe, fulfils with N

NeMo Helix: Make the agents you ship faster, more accurate, and safer.

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Your reading list, as a podcast you own

Distill teacher chains-of-thought into a LoRA adapter via a strict boxed-answer format contract + two-phase Train→Nudge (silver-medal NVIDIA

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

Production-ready, bleeding-edge, upstream-first vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a), verified against real models on

Hermes Agent (NousResearch) sandboxed by NVIDIA OpenShell — hardware-enforced network/filesystem/syscall policy, full memory + gateway stack

OpenAI-compatible gateway aggregating 8 free LLM providers with automatic failover, circuit breakers, and smart routing. 32+ models, zero c

📊 Automated hourly benchmarks for 20+ NVIDIA NIM models — interactive dashboard, zero infra, self-hostable. Open-source & community-driven.

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Browser-based model and inference management for NVIDIA DGX Spark - inventory local and Hugging Face models, manage Ollama and LiteLLM, gene

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding

CPU-side action routing, context compression, and causal memory for AI agents — matches an LLM-everything agent at 58% fewer LLM calls and ~

High-performance CUDA inference engine for Qwen models, written from scratch in C++/CUDA

Dashboard for InferenceX™, Open Source Continuous Inference | InferenceX 仪表板

让 Codex 白嫖 NVIDIA 免费顶级模型，一键接入 | Transparent proxy between Codex and NVIDIA NIM — use NVIDIA's free hosted top-tier models with Codex

Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

The ultimate memory backend for OpenClaw and Claude Code. Persistent conversation memory with LLM fact extraction, foundation-model rerankin

Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simula

See every GPU in your lab in the terminal, the browser or a report — who holds each GPU, which ones sit idle. One Python file, nothing to in

The visual compute platform for AI and robotics — connecting agents, GPU workloads, simulation, training, and real robots through typed work

Zero-dependency Go proxy for rotating free-tier AI API keys. No more 429s.

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel e

The old advice to avoid Windows for local LLMs used to be right. It isn't anymore. The fixes for display-GPU desktop crashes and WSL memory

Realtime TUI monitor for local LLM servers (ollama, llama.cpp). The only GPU monitor that knows what model is running and how much each toke