Havuz

Laplace's Scissors: protected readout rotations for open LLMs. Diagnose weak output directions, repair answers, preserve successful generati

Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX cards (tuned on 3080 10gb). A measured, tuned and benchmarked fork of Str

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

Local-first AI video dubbing pipeline: transcription, translation, speech synthesis and timing alignment.

🔥🔥 Accelerating decision inference with exact state forking

Turkish voice design with VoxCPM2 on a 4 GB GPU: describe a voice, pick a variant, read text in 30 languages. Patched audio.cpp, local UI in

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Private, offline AI assistant for Android and Windows. Run local LLMs (Gemma 4, Qwen, GGUF) on your phone or PC with llama.cpp, CUDA and Vul

Windows utility focused on cuda setup, nvidia local AI, and cuda environment for local AI workflows.

Local AI BGM and sound effects for Claude Code and any MCP client. Stable Audio 3 on Apple Silicon (MLX) and Windows NVIDIA GPUs (PyTorch).

Run Qwen3.8-Flash-Next (125B) on an RTX 3060 12 GB with 30 GB of RAM, streaming experts from the SSD. 50-70 tok/s, 262K context. A fork of S

Qwen Image 2.1 for DeepSeek Harness: native image editing on Apple Silicon MLX/MFLUX or self-hosted CUDA, LoRA, ControlNet, inpainting, pres

Qwen3.8-Flash-Next (180B MoE) on one 12-16 GB NVIDIA GPU, Linux or Windows. Own engine, own compression.

Pure-Rust MoE inference engine for Qwen3.5/3.6/3.8 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp

A tool that allows AI to direct the dubbing process, uses the original actor voices and does the audio generation on your local PC

Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and

The fastest inference engine for Qwen3.8-27B on the NVIDIA RTX 5090: up to 500 tokens/s for one agent and up to 2,000 tokens/s for many, con

Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVID

Local, offline text-to-speech and voice cloning on audio.cpp — multi-voice narration, voice design, sound/music generation, and a real-time

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

A realtime assistant for Mac and Windows. Talk to it and it answers aloud, shows your schedule and mail as cards, and hands longer work to a

OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.

Model Dispatch and Memory Allocator: an LLM inference engine for the hardware you own. Custom CPU, ROCm, Vulkan, CUDA, Metal and Level Zero

Glyd, model ağırlıklarını ve KV cache'i bit düzeyinde kayıpsız biçimde sıkıştırıp GPU'da kullanmayı hedefleyen bir compression yazılımıdır.

Neden yükseliyor · README, yaygın bir maliyet sorununa doğrudan yanıt verdiğini ve 32B ile 72B modellerde daha az GPU ile çalışmaya dair ölçümler sunuyor; ancak 9 günlük repoda 170 yıldız ve 474 commit'in yalnızca 2 farklı yazardan gelmesi, ilginin ve geliştirme temposunun geniş bir katkıcı tabanına dayandığını göstermiyor. Yükselişin kesin kaynağı veya bağımsız benimsenme verisi bilinmiyor.

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

Local LLMs that run lean, load instantly and keep getting smarter. A from-scratch runtime + model format in Zig: zero-copy mmap, CPU/CUDA/Vu

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

High-performance CUDA inference engine for Qwen models, written from scratch in C++/CUDA

llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and be

An inference engine for one NVIDIA RTX card, built for a coding agent and its subagents, and a companion that listens, reads and draws: Qwen

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

🧱 Experimental library for building inference engines

llama.cpp performance patches developed and measured on Tesla P100 (GP100, sm_60). Most are not Pascal-specific.

Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen

AI/LLM on Kubernetes 基础设施知识库：涵盖 GPU、大数据、调度、分布式训练、LLM 推理、RAG、Agent 与生产运维。| AI/LLM on Kubernetes Infrastructure Knowledge Base: Covering GPUs,

Run LLMs and Ollama Qwen across AMD XDNA1 NPU, CPU, and GPU on Ryzen AI Linux systems.

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

Implementation of LatentMoE,Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts (Elango et al., NVIDIA 2026) — in Pytorch.

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

Memory-aware GGUF inference runtime with OpenAI-compatible serving

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, depl

Tiny engine, immense models — run large MoE LLMs (gpt-oss, Mixtral, Qwen3-MoE) on ordinary machines by streaming experts from disk. OpenAI-c

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, b

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig