Havuz

TurboFieldfare, Gemma 4 26B-A4B modelini Apple Silicon Mac’lerde Swift ve Metal tabanlı özel bir runtime ile yerel olarak çalıştırır.

Neden yükseliyor · 72 günde 6828 yıldız alması ve 8 GB RAM’li Mac’te büyük bir model çalıştırma iddiası dikkat çekiyor. README belirli cihazlarda ölçülmüş decode hızları ve deney kayıtları sunuyor; ancak yıldızlar benimseme veya kalite kanıtı değil. Son 28 günde 4 commit ve 1 farklı yazar olması, geliştirme etkinliği ve ekip genişliği açısından sınırlı kanıt sağlıyor.

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Slotstream, büyük bir MoE modelini SSD’den parça parça yükleyip yoğun kullanılan expert’leri bellekte önbelleğe alarak Apple Silicon Mac’lerde çalıştıran yerel bir inference engine’dir.

Neden yükseliyor · README, 48 GB M5 Pro üzerinde belirli koşullarda 15.86 tok/s ölçümü ve prompt işleme iyileştirmeleri yayımlıyor; son 28 günde 271 commit ve 5 farklı yazar da yoğun geliştirme faaliyetine işaret ediyor. Ancak proje yalnızca 30 günlük ve mevcut sürüm için nitelikli tam yanıt hızı ölçümünün hâlâ beklediği belirtiliyor.

Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache co

Open-source local AI workspace — advancing on-device inference.

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

LLM serving engines that SkyDiscover builds on the fly for specific AI models and hardware: Qwen3.6-35B-A3B, Qwen3.8-Flash-Next and Flint on

Real-time multimodal AI pipelines on Apple Silicon

Run a coding model at 300+ tok/s on your Mac. Native Rust + Metal inference engine for LiquidAI LFM2.5-24B-A2B (MLX 4-bit) on Apple Silicon.

Run large language models locally on Intel Macs with AMD GPUs - native macOS app with Metal acceleration

DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any M-series MacBook

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memo

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.

DeepSeek-V4.1-Flash-1017 on one 256 GB M5 Ultra: 1M context, 70.7-104.9 tok/s decode with DSpark drafts, 1,739-1,867 tok/s cold prompts (8k-

Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

Model Dispatch and Memory Allocator: an LLM inference engine for the hardware you own. Custom CPU, ROCm, Vulkan, CUDA, Metal and Level Zero

Interactive terminal coding agent in Rust — runs the DeepSeek V4 Flash model locally on macOS (Metal), with a Ratatui TUI, a full tool suite

Apple Silicon mlx with Zero Dependency Java

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen

The Kimi K3 and DeepSeek-V4-Flash-0731 architectures, reimplemented from scratch in pure Rust (zero dependencies) and verified 1:1 against t

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, b

Memory-aware GGUF inference runtime with OpenAI-compatible serving

ML model ports (ASR · TTS · LLM · VLM · vision · audio codecs) on the rlx multi-backend compiler + runtime — CPU/Metal/MLX/CUDA/wgpu

TurboQuant MLX: Flagship high-performance model quantization for Apple Silicon. Squeeze maximum inference speed and memory efficiency out of

🧠 One LLM, split across a Mac (Apple MPS) + a Windows PC (NVIDIA CUDA) — heterogeneous pipeline inference over a framework-neutral wire, bit

Fast KV-cache quantization for Apple Silicon (MLX) — 43 research-adapted compression methods with Metal kernels

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

neural networks in pure C

Transformer inference via unconventional compute: a single algebraic primitive, a tag-system scheduler, and Apple's Neural Engine through co

Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddings

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.