Havuz

kimi-k3-in-c, Kimi K3 modelinde inference çalıştırmayı hedefleyen bağımlılıksız bir C99 motorudur.

Neden yükseliyor · 57 günde 8.683 yıldız alması ve Kimi K3'ü 8 GB RAM'li CPU'da çalıştırma iddiası dikkat çekiyor; ancak son 28 gündeki 12 commit'in tek yazardan gelmesi, ilginin bağımsız teknik doğrulama veya geniş katkıcı tabanıyla desteklendiğini göstermiyor.

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU

PROMETHEVS: run the full DeepSeek-V4.1-Flash-1017 (510 GB) on a gaming PC with 64 GB RAM and a 12 GB GPU. PROMETHEVS engine (compress-then-c

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cp

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s

turbovec, TurboQuant ile vektörleri sıkıştırıp Rust ve Python üzerinden arama yapmayı sağlayan yerel bir vector index kütüphanesidir.

Neden yükseliyor · 185 günde 17.245 yıldız almış olması ve bellek tasarrufu ile hızlı yerel arama vaadi dikkat çekiyor. Ancak son 28 günde commit ve farklı yazar sayısı sıfır; bu veriler yükselişin güncel olarak sürdüğünü göstermiyor. README'deki performans karşılaştırmaları proje tarafından bildiriliyor, bağımsız doğrulama bilgisi verilen ölçümlerde yok.

Live English speech recognition on the ESP32-P4: a 65.7M-parameter streaming transducer in 2/4-bit weights, faster than real time, on the ch

Bilingual (中文+EN) ML / LLM / diffusion / agent interview cheat sheets for AI 秋招 — generated by ARIS /interview-cheatsheet, rendered by /rend

Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

Kyojin: open-source engine that runs the largest open models on one mini PC or one graphics card, about 2× faster than llama.cpp. AMD (ROCm)

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that d

一套功能完整、轻量稳定的 ComfyUI 实用节点集。初衷是为了平替一些断更插件的功能，会持续更新一些新开发的节点插件。有好的创意可以发Issues，我会力所能及想办法做出来。全中文界面，原生控件，不依赖第三方库。

ScribeCrate: open-source, self-hosted Whisper speech-to-text server powered by faster-whisper. OpenAI-compatible transcription and English t

Halo is an open-source framework built by White Circle for training large language and multimodal models

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apach

Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE

600 KB WASM runtime for Cactus Compute's Needle AI tool-calling models, Needle 3, 2 and 1 from one build. Browser, Cloudflare Workers, Node.

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kern

Optimized ESP32-S3/P4 fully offline speech to text and text to speech system for Micropython and AtomVM, featuring a finetuned STT model for

Fast KV-cache quantization for Apple Silicon (MLX) — 43 research-adapted compression methods with Metal kernels

[ICML 2026] NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

Memory backbone for AI agents. Four-tier store — working, episodic, semantic, procedural — with decay, consolidation, and association graphs

Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddings

DeepSeek-V4.1-Flash-1017 on one 256 GB M5 Ultra: 1M context, 70.7-104.9 tok/s decode with DSpark drafts, 1,739-1,867 tok/s cold prompts (8k-

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Python package for LLM compression

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

QuantClaw is a plug-and-play task-type routing quantization plugin for OpenClaw.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

Honest local LLM deployment planning and benchmarking for high unified-memory Macs and future Linux/NVIDIA rigs.

Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant)

Open source skill library for AI coding agents to write, optimize, and debug high performance compute kernels across CUDA, Triton, and quant

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression wit

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Interactive canvas for taking an LLM apart tensor by tensor: measured INT8/INT4/FP8 error, distributions, spectra and outlier channels for e

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, depl

559 K パラメータの日本語 TTS を ESP32-S3 で実時間合成。漢字かな交じり文の形態素解析・アクセント推定まで端末内で走る（M5Stack CoreS3 実機で確認）。推論は依存ゼロの C99、ブラウザ demo あり。arXiv:2608.21378 sanoTT

⚡ In-Memory Vector Search Engine & HNSW ANN Graph in Java 21 Loom. Sub-millisecond KNN retrieval, 8-bit scalar quantization (SQ8), 16,000+ Q