Havuz

turbovec, TurboQuant ile vektörleri sıkıştırıp Rust ve Python üzerinden arama yapmayı sağlayan yerel bir vector index kütüphanesidir.

Neden yükseliyor · 185 günde 17.245 yıldız almış olması ve bellek tasarrufu ile hızlı yerel arama vaadi dikkat çekiyor. Ancak son 28 günde commit ve farklı yazar sayısı sıfır; bu veriler yükselişin güncel olarak sürdüğünü göstermiyor. README'deki performans karşılaştırmaları proje tarafından bildiriliyor, bağımsız doğrulama bilgisi verilen ölçümlerde yok.

kimi-k3-in-c, Kimi K3 modelinde inference çalıştırmayı hedefleyen bağımlılıksız bir C99 motorudur.

Neden yükseliyor · 57 günde 8.683 yıldız alması ve Kimi K3'ü 8 GB RAM'li CPU'da çalıştırma iddiası dikkat çekiyor; ancak son 28 gündeki 12 commit'in tek yazardan gelmesi, ilginin bağımsız teknik doğrulama veya geniş katkıcı tabanıyla desteklendiğini göstermiyor.

Bilingual (中文+EN) ML / LLM / diffusion / agent interview cheat sheets for AI 秋招 — generated by ARIS /interview-cheatsheet, rendered by /rend

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Python package for LLM compression

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apach

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

QuantClaw is a plug-and-play task-type routing quantization plugin for OpenClaw.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE

600 KB WASM runtime for Cactus Compute's Needle AI tool-calling models, Needle 3, 2 and 1 from one build. Browser, Cloudflare Workers, Node.

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant)

Open source skill library for AI coding agents to write, optimize, and debug high performance compute kernels across CUDA, Triton, and quant

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression wit

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Interactive canvas for taking an LLM apart tensor by tensor: measured INT8/INT4/FP8 error, distributions, spectra and outlier channels for e

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, depl

Open-source platform for creating, distributing and running sovereign EU-compliant LLMs. Verticalize any model for your domain, language and

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kern

llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.

🏆Winning Project | ModelGate is a contract-aware AI control plane that ingests customer contracts, extracts SLA/privacy/routing constraints,

Run a 28.9M-parameter TinyLM on ESP32-S3 with an RP2040 OLED display node for fully local embedded AI inference.

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

TurboQuant MLX: Flagship high-performance model quantization for Apple Silicon. Squeeze maximum inference speed and memory efficiency out of

Rust inference engine for a 28.9M-parameter PLE TinyLM on the ESP32-S3 — 9.4 tok/s, faster than the C implementation it was ported from

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

Run AI models on Apple Silicon and Raspberry Pi

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

A native macOS lab for teaching tiny language models to think — build the architecture, train the weights, and watch a small LLM emerge from

Open-source vector database and sub-microsecond key-value store in one Go engine — embed it as a library, run it standalone, or replicate it

Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, re

vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen