Havuz

A native macOS terminal built in Swift, with Metal rendering, CJK input, and a dependency-free terminal core.

A free textbook on building an LLM inference stack from scratch in Rust.

How many simultaneous 4K transcodes can your GPU sustain? Multi-vendor (Intel/AMD/NVIDIA/CPU) benchmark for Unraid with HDR tone-mapping, su

PantheonSim: GPU simulator for CUDA and ROCm. Run CUDA and HIP programs unmodified on a CPU, no GPU needed: simulated NVIDIA and AMD GPUs fo

Self-hosted meeting transcription portal — speech-to-text, speaker diarization, LLM-corrected transcripts, structured summaries and Word min

Eve Agent V2 Unleashed — local-first autonomous AI coding agent powered by Ollama

`roctop` is a small AMD ROCm GPU monitor. It shows GPU VRAM usage, utilization, temperature, and GPU processes in a refreshing terminal UI.

Native macOS menu bar app for real-time CPU, GPU, temperature, power and fan monitoring — Apple Silicon and Intel.

CLI-native local AI image and video generation for people, scripts, and agents — CUDA on Linux, Metal on macOS, plus desktop, web, iPhone, R

MLOps — vLLM + LiteLLM + Portainer CE + Netdata. Case study song ngữ: kiến trúc, metrics, bài học vận hành. · Self-hosted LLM inference don

Overclock or undervolt your NVIDIA RTX GPU from the command line — a no-background-process MSI Afterburner alternative for Windows.

Apache Arrow compute on Apple silicon GPUs through Metal: Arrow columns in unified memory, copy-free where the producer's buffers are page a

Which local LLMs can your computer run? Hardware-aware CLI, TUI and GUI: yes/slow/no verdicts and tok/s from your GPU VRAM and RAM, then ver

Public home for Metalterm — a GPU-rendered terminal for macOS. Downloads, release notes, bug reports, and feature requests

Rust game engine for huge physics scenes: 10,000 GPU-simulated cubes at 2,000+ FPS on an RTX 3060. Vulkan, editor, deterministic replay, age

One GPU. Full LLM workflow. Real benchmarks. No cloud required.

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

Legacy: self-hosted LLM/RAG stack installer for DGX Spark (arm64). The work continued as AGmind Systems Lab — measured local-LLM performance

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → paralleli

A narrative walk through 90 years of AI history, paper by paper. 66 chapters from Turing 1936 to Blackwell 2025.

One command spins up Qwen3.8-27B (abliterated, NVFP4) on a rented Blackwell GPU at ~150 tok/s and wires it into your local coding agent via

A highly compliant C++ gpu accelerated html5 rendering engine as a (mostly) single header C++ zero-dependency lib - designed to run real-wor

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Immediate-mode app platform — an Electron alternative with real GPU graphics. C++ 2D/3D renderer with optional Dawn/WebGPU backend, embedded

🔥 AI agent that ports CUDA code to AMD ROCm using local LLMs — FastAPI + React + Ollama | Live: rocmporter-agent.vercel.app

Operator-grade GPU monitor for NVIDIA GPUs with native GB10 / DGX Spark coherent UMA support — PSI pressure, clock detection, ConnectX-7 net

Your all-in-one funscript editor, generator, and player. Free tracking algorithms. A Pro Pass unlocks the next level AI generation right on

Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

SpeakCrate: self-hosted text-to-speech API powered by Kokoro. OpenAI-compatible speech endpoint, 50+ voices across 9 languages, streamed aud

Honeycomb Lab — hex map + OpenAI gateway control plane for a home AI fleet

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simula

See every GPU in your lab in the terminal, the browser or a report — who holds each GPU, which ones sit idle. One Python file, nothing to in

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a lapt

The visual compute platform for AI and robotics — connecting agents, GPU workloads, simulation, training, and real robots through typed work

One CLI to search, provision, and manage cloud GPUs across 10 providers

Local-first AI assistant for Windows: voice (Whisper+Piper), webcam vision, FLUX image gen, image-to-3D, memory, 80+ tools. OpenAI-compatibl

Grammar-aware adaptive speculative decoding for structured LLM output — custom CUDA/Triton kernels, 2.7× throughput on enum-heavy schemas

nvtop for vLLM — an interactive terminal dashboard for vLLM serving performance (concurrency, throughput, cache & KV memory, latency, spec-d

Safety-first self-hosted FFmpeg media-library optimiser for Plex, Jellyfin, and Emby, with verified CPU/GPU transcoding and rollback.

nvitop-like terminal monitor for MetaX GPUs

LLM Infra 工程实战 — 面向应用层工程师的 LLM 基础设施

MCP server for NVIDIA GPU metrics give AI agents real-time access to GPU utilization, memory, temperature, and power

A Metal-rendered creative-coding framework for Swift on Apple platforms: live reload and live coding, generative techniques, GPU shaders and

A self-hosted, AWS-compatible cloud for your own hardware. Point the AWS CLI, SDKs, CloudFormation and Terraform at it to run Lambda, S3, RD

Profine automatically profiles and optimizes PyTorch training jobs on real GPUs, delivering measurable speedups and lower GPU costs before t

Kata builds the best AI agent for a subnet through open competition — so anyone can mine that subnet with a proven, optimized agent.

The terminal your agent team lives in — run coding agents in panes on a shared bus, on a fast GPU-rendered terminal for macOS and Linux.

Multi-GPU prefill acceleration for llama.cpp

Pure-Rust native GUI framework with web-grade aesthetics and first-class headless PNG rendering — winit + wgpu + vello + parley + taffy