Havuz

Grammar-aware adaptive speculative decoding for structured LLM output — custom CUDA/Triton kernels, 2.7× throughput on enum-heavy schemas

Kata builds the best AI agent for a subnet through open competition — so anyone can mine that subnet with a proven, optimized agent.

MCP server for NVIDIA GPU metrics give AI agents real-time access to GPU utilization, memory, temperature, and power

Fastest MoE/LLM inference runtime for consumer and edge Blackwell GPUs. SN74 on Gittensor.

Local AI agent sandbox runtime with Firecracker, gVisor, and NVIDIA GPU sandbox support.

`roctop` is a small AMD ROCm GPU monitor. It shows GPU VRAM usage, utilization, temperature, and GPU processes in a refreshing terminal UI.

Overclock or undervolt your NVIDIA RTX GPU from the command line — a no-background-process MSI Afterburner alternative for Windows.

A self-hosted, AWS-compatible cloud for your own hardware. Point the AWS CLI, SDKs, CloudFormation and Terraform at it to run Lambda, S3, RD

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

The remote-native terminal: your shell, the machine it runs on, and its files in one window. Durable SSH sessions, remote files, GPU and por

OpenAI-compatible multilingual TTS server — Chatterbox on vLLM with real-time PCM audio streaming, low time-to-first-byte (~0.7 s), voice cl

The terminal your agent team lives in — run coding agents in panes on a shared bus, on a fast GPU-rendered terminal for macOS and Linux.

nvtop for vLLM — an interactive terminal dashboard for vLLM serving performance (concurrency, throughput, cache & KV memory, latency, spec-d

Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a lapt

Pure-Rust native GUI framework with web-grade aesthetics and first-class headless PNG rendering — winit + wgpu + vello + parley + taffy

SiliconScope, Apple Silicon Mac’lerde sistem kaynaklarını ve özel hızlandırıcıları izleyen, ayrıca uzaktaki makineleri aynı panele ekleyebilen bir macOS monitörüdür.

Neden yükseliyor · 111 günde 966 yıldıza ulaşması ve son 28 günde 48 commit alması dikkat çekiyor; README’deki ANE ve memory-bandwidth ölçümleriyle uzaktaki makineleri tek panelde toplama özellikleri de ayırt edici bir kullanım vaadi sunuyor. Üç farklı yazar aktiviteye işaret etse de ekip genişliği sınırlı; yıldızlar tek başına benimseme veya kalite kanıtı değil.

Safety-first self-hosted FFmpeg media-library optimiser for Plex, Jellyfin, and Emby, with verified CPU/GPU transcoding and rollback.

Agent-friendly GPU profile-query CLI

Multi-GPU prefill acceleration for llama.cpp

Local-first AI assistant for Windows: voice (Whisper+Piper), webcam vision, FLUX image gen, image-to-3D, memory, 80+ tools. OpenAI-compatibl

Native macOS terminal with GPU rendering, durable sessions, coding-agent workflows, and SSH connectivity. Session host, daemon, and CLI supp

Turn any document or a whole zip into an interactive knowledge graph, using a self-hosted Qwen3.6-35B-A3B-MTP on a single NVIDIA L4

Sleek, mobile-friendly web UI for NVIDIA LocateAnything-3B — open-vocabulary object detection & grounding on your own GPU, via one docker co

Your all-in-one funscript editor, generator, and player. Free tracking algorithms. A Pro Pass unlocks the next level AI generation right on

Self-hosted homelab dashboard in one container — GPU, local-AI VRAM, Docker, systemd, energy costs. Read-only MCP server so AI agents can ex

LLM Infra 工程实战 — 面向应用层工程师的 LLM 基础设施

Eve Agent V2 Unleashed — local-first autonomous AI coding agent powered by Ollama

A Metal-rendered creative-coding framework for Swift on Apple platforms: live reload and live coding, generative techniques, GPU shaders and

A highly compliant C++ gpu accelerated html5 rendering engine as a (mostly) single header C++ zero-dependency lib - designed to run real-wor

The visual compute platform for AI and robotics — connecting agents, GPU workloads, simulation, training, and real robots through typed work

CCO is an open-source system for managing, validating, and improving GPU.

Single-GPU LLM fine-tuning preflight: memory estimates, runnable LoRA/QLoRA recipes, and local measurements.

One GPU. Full LLM workflow. Real benchmarks. No cloud required.

Self-hosted meeting transcription portal — speech-to-text, speaker diarization, LLM-corrected transcripts, structured summaries and Word min

GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% wi

Profine automatically profiles and optimizes PyTorch training jobs on real GPUs, delivering measurable speedups and lower GPU costs before t

ParseCrate: self-hosted document processing API powered by Docling and docling-serve. Converts PDF, DOCX, PPTX, HTML, and more to Markdown/J

nvitop-like terminal monitor for MetaX GPUs

Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

Immediate-mode app platform — an Electron alternative with real GPU graphics. C++ 2D/3D renderer with optional Dawn/WebGPU backend, embedded

Realtime TUI monitor for local LLM servers (ollama, llama.cpp). The only GPU monitor that knows what model is running and how much each toke

OpenLake, GPU tabanlı LLM inference ve training iş yükleri için dağıtık, kalıcı ve yüksek throughput hedefleyen bir storage engine'dir.

Neden yükseliyor · 154 günlük bir repo için 2.658 yıldız dikkat çekici; KV cache offload ve GPU kullanımını artırma gibi güncel, maliyetle doğrudan ilişkili bir altyapı sorununa odaklanıyor. Son 28 günde 16 commit var, ancak bunlar yalnızca 3 farklı yazardan geliyor. README'deki benchmark ve performans iddiaları proje tarafından yayımlanmış; verilen bilgiler bağımsız doğrulamayı göstermiyor.

A GPU-rendered, terminal-first workspace: Agent framework, real shells, IDE, Markdown, and multiplayer — one Rust engine.

One CLI to search, provision, and manage cloud GPUs across 10 providers

A narrative walk through 90 years of AI history, paper by paper. 66 chapters from Turing 1936 to Blackwell 2025.

A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Operator-grade GPU monitor for NVIDIA GPUs with native GB10 / DGX Spark coherent UMA support — PSI pressure, clock detection, ConnectX-7 net

The Best TypeScript framework for fine-tuning