Havuz

kimi-k3-in-c, Kimi K3 modelinde inference çalıştırmayı hedefleyen bağımlılıksız bir C99 motorudur.

Neden yükseliyor · 57 günde 8.683 yıldız alması ve Kimi K3'ü 8 GB RAM'li CPU'da çalıştırma iddiası dikkat çekiyor; ancak son 28 gündeki 12 commit'in tek yazardan gelmesi, ilginin bağımsız teknik doğrulama veya geniş katkıcı tabanıyla desteklendiğini göstermiyor.

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cp

halogen-flash-server, Qwen3.8-Flash-Next modelini AMD Strix Halo (gfx1151) donanımında çalıştırmak için özelleştirilmiş bir inference server'dır.

Neden yükseliyor · 32 günlük repo için 722 yıldız ve son 28 günde 99 commit dikkat çekiyor. README, belirli donanım üzerindeki ölçümleri ve ölçüm koşullarını açıklıyor; ancak performans iddiaları proje tarafından sunulmuş ve bağımsız doğrulama bilgisi burada yok. 99 commit'in tek yazardan gelmesi geliştirme hızını gösterirken bakım kapasitesine dair kanıtı sınırlıyor.

Slotstream, büyük bir MoE modelini SSD’den parça parça yükleyip yoğun kullanılan expert’leri bellekte önbelleğe alarak Apple Silicon Mac’lerde çalıştıran yerel bir inference engine’dir.

Neden yükseliyor · README, 48 GB M5 Pro üzerinde belirli koşullarda 15.86 tok/s ölçümü ve prompt işleme iyileştirmeleri yayımlıyor; son 28 günde 271 commit ve 5 farklı yazar da yoğun geliştirme faaliyetine işaret ediyor. Ancak proje yalnızca 30 günlük ve mevcut sürüm için nitelikli tam yanıt hızı ölçümünün hâlâ beklediği belirtiliyor.

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimiz

Automated alignment adjustment for LLMs — direct steering, LoRA, and MoE expert-granular abliteration, optimized via multi-objective Optuna

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

Live truecolor terminal dashboard for a locally running LLM: tok/s, TTFT, GPUs, context, MTP acceptance, layer and real expert routing (llam

ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verifi

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

Fastest MoE/LLM inference runtime for consumer and edge Blackwell GPUs. SN74 on Gittensor.

The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memo

⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

hummingbird is a lightweight, zero dependency runtime for massive open source Mixture of Experts (MoE) language models. It unifies SSD, RAM,

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kern

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, b

Distill teacher chains-of-thought into a LoRA adapter via a strict boxed-answer format contract + two-phase Train→Nudge (silver-medal NVIDIA

Instella MoE is a fully open state-of-the-art Mixture-of-Experts language model, trained end-to-end on AMD Instinct™ GPUs.

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, depl

MLX-compatible REAP for pruning MoE models on Apple Silicon

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

Self-hosted, browser-based AI dungeon RPG, fully local on an AMD Strix Halo APU: narrator + per-character agents on one local LLM (Gemma 4 2

Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, re

The Kimi K3 and DeepSeek-V4-Flash-0731 architectures, reimplemented from scratch in pure Rust (zero dependencies) and verified 1:1 against t

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

A user's facts become a few rows in a content-addressed memory table — not a rewrite of the model. Paper, code, data, and interactive site f

Run models too big for RAM on Apple Silicon — any Hugging Face model, dense or MoE — by streaming weights from disk

👻 MoEWatch: A lightweight diagnostic and audit library for Mixture-of-Experts (MoE) models in HuggingFace Transformers — the pytest for MoE

Open benchmark for expert collapse and routing efficiency in sparse MoE LLMs

Build and train a mini Kimi K3 from scratch — a pure-PyTorch playground from a custom 88M model to 2T-scale LLM profiles.

Tiny engine, immense models — run large MoE LLMs (gpt-oss, Mixtral, Qwen3-MoE) on ordinary machines by streaming experts from disk. OpenAI-c

A compact, readable PyTorch reference implementation of the DeepSeek-V4 architecture (Flash and Pro), in the spirit of nanoGPT. ~4,300 LOC.