Havuz

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU

Don't memorize ML. Understand it from first principles including Inference, Cuda, Deep Learning, Data Science and Reinforcement Learning

Glyd, model ağırlıklarını ve KV cache'i bit düzeyinde kayıpsız biçimde sıkıştırıp GPU'da kullanmayı hedefleyen bir compression yazılımıdır.

Neden yükseliyor · README, yaygın bir maliyet sorununa doğrudan yanıt verdiğini ve 32B ile 72B modellerde daha az GPU ile çalışmaya dair ölçümler sunuyor; ancak 9 günlük repoda 170 yıldız ve 474 commit'in yalnızca 2 farklı yazardan gelmesi, ilginin ve geliştirme temposunun geniş bir katkıcı tabanına dayandığını göstermiyor. Yükselişin kesin kaynağı veya bağımsız benimsenme verisi bilinmiyor.

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark,

Qwen Image 2.1 for DeepSeek Harness: native image editing on Apple Silicon MLX/MFLUX or self-hosted CUDA, LoRA, ControlNet, inpainting, pres

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

Sleek, mobile-friendly web UI for NVIDIA LocateAnything-3B — open-vocabulary object detection & grounding on your own GPU, via one docker co

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document parsing, embeddings, transcription, text-to-speech and MCP too

GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% wi

Agent-friendly GPU profile-query CLI

OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at

Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.

Evaluating frontier models on real-world ai-infra engineering tasks

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

ScribeCrate: open-source, self-hosted Whisper speech-to-text server powered by faster-whisper. OpenAI-compatible transcription and English t

Model Dispatch and Memory Allocator: an LLM inference engine for the hardware you own. Custom CPU, ROCm, Vulkan, CUDA, Metal and Level Zero

PantheonSim: GPU simulator for CUDA and ROCm. Run CUDA and HIP programs unmodified on a CPU, no GPU needed: simulated NVIDIA and AMD GPUs fo

CLI-native local AI image and video generation for people, scripts, and agents — CUDA on Linux, Metal on macOS, plus desktop, web, iPhone, R

ScribeCrate Live: self-hosted real-time transcription server powered by WhisperLive and faster-whisper. WebSocket transcription and speech-t

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

🔥 AI agent that ports CUDA code to AMD ROCm using local LLMs — FastAPI + React + Ollama | Live: rocmporter-agent.vercel.app

Operator-grade GPU monitor for NVIDIA GPUs with native GB10 / DGX Spark coherent UMA support — PSI pressure, clock detection, ConnectX-7 net

SpeakCrate: self-hosted text-to-speech API powered by Kokoro. OpenAI-compatible speech endpoint, 50+ voices across 9 languages, streamed aud

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

ParseCrate: self-hosted document processing API powered by Docling and docling-serve. Converts PDF, DOCX, PPTX, HTML, and more to Markdown/J