Havuz

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU

kimi-k3-in-c, Kimi K3 modelinde inference çalıştırmayı hedefleyen bağımlılıksız bir C99 motorudur.

Neden yükseliyor · 57 günde 8.683 yıldız alması ve Kimi K3'ü 8 GB RAM'li CPU'da çalıştırma iddiası dikkat çekiyor; ancak son 28 gündeki 12 commit'in tek yazardan gelmesi, ilginin bağımsız teknik doğrulama veya geniş katkıcı tabanıyla desteklendiğini göstermiyor.

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cp

Kyojin: open-source engine that runs the largest open models on one mini PC or one graphics card, about 2× faster than llama.cpp. AMD (ROCm)

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

turbovec, TurboQuant ile vektörleri sıkıştırıp Rust ve Python üzerinden arama yapmayı sağlayan yerel bir vector index kütüphanesidir.

Neden yükseliyor · 185 günde 17.245 yıldız almış olması ve bellek tasarrufu ile hızlı yerel arama vaadi dikkat çekiyor. Ancak son 28 günde commit ve farklı yazar sayısı sıfır; bu veriler yükselişin güncel olarak sürdüğünü göstermiyor. README'deki performans karşılaştırmaları proje tarafından bildiriliyor, bağımsız doğrulama bilgisi verilen ölçümlerde yok.

llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s

Bilingual (中文+EN) ML / LLM / diffusion / agent interview cheat sheets for AI 秋招 — generated by ARIS /interview-cheatsheet, rendered by /rend

Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).

STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

Halo is an open-source framework built by White Circle for training large language and multimodal models

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Python package for LLM compression

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

ScribeCrate: open-source, self-hosted Whisper speech-to-text server powered by faster-whisper. OpenAI-compatible transcription and English t

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

600 KB WASM runtime for Cactus Compute's Needle AI tool-calling models, Needle 3, 2 and 1 from one build. Browser, Cloudflare Workers, Node.

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apach

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

QuantClaw is a plug-and-play task-type routing quantization plugin for OpenClaw.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous