Havuz

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the ha

Slotstream, büyük bir MoE modelini SSD’den parça parça yükleyip yoğun kullanılan expert’leri bellekte önbelleğe alarak Apple Silicon Mac’lerde çalıştıran yerel bir inference engine’dir.

Neden yükseliyor · README, 48 GB M5 Pro üzerinde belirli koşullarda 15.86 tok/s ölçümü ve prompt işleme iyileştirmeleri yayımlıyor; son 28 günde 271 commit ve 5 farklı yazar da yoğun geliştirme faaliyetine işaret ediyor. Ancak proje yalnızca 30 günlük ve mevcut sürüm için nitelikli tam yanıt hızı ölçümünün hâlâ beklediği belirtiliyor.

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

Run large language models locally on Intel Macs with AMD GPUs - native macOS app with Metal acceleration

Native macOS terminal with GPU rendering, durable sessions, coding-agent workflows, and SSH connectivity. Session host, daemon, and CLI supp

Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built

Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml: Qwen3-TTS, Irodori-TTS, Qwen3-ASR,

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache co

DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any M-series MacBook

Real-time multimodal AI pipelines on Apple Silicon

Lightning-fast, free, local first voice dictation for macOS with on-device transcription

Open-source local AI workspace — advancing on-device inference.

TurboFieldfare, Gemma 4 26B-A4B modelini Apple Silicon Mac’lerde Swift ve Metal tabanlı özel bir runtime ile yerel olarak çalıştırır.

Neden yükseliyor · 72 günde 6828 yıldız alması ve 8 GB RAM’li Mac’te büyük bir model çalıştırma iddiası dikkat çekiyor. README belirli cihazlarda ölçülmüş decode hızları ve deney kayıtları sunuyor; ancak yıldızlar benimseme veya kalite kanıtı değil. Son 28 günde 4 commit ve 1 farklı yazar olması, geliştirme etkinliği ve ekip genişliği açısından sınırlı kanıt sağlıyor.

The agentic node editor for live visual effects

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.

TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.

The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memo

A practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device inference.

Iris-3B pixel-space text-to-image on Apple Silicon GPUs: Metal-optimized engine, native macOS app, CLI, Python and local HTTP API

CLI-native local AI image and video generation for people, scripts, and agents — CUDA on Linux, Metal on macOS, plus desktop, web, iPhone, R

Apple Silicon mlx with Zero Dependency Java

TurboQuant MLX: Flagship high-performance model quantization for Apple Silicon. Squeeze maximum inference speed and memory efficiency out of

Free, open-source screen recorder for macOS. A share link the moment you stop, then auto-zoom, smooth cursor and captions under the same lin

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

ML model ports (ASR · TTS · LLM · VLM · vision · audio codecs) on the rlx multi-backend compiler + runtime — CPU/Metal/MLX/CUDA/wgpu

Interactive terminal coding agent in Rust — runs the DeepSeek V4 Flash model locally on macOS (Metal), with a Ratatui TUI, a full tool suite

Fast KV-cache quantization for Apple Silicon (MLX) — 43 research-adapted compression methods with Metal kernels

neural networks in pure C

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, b

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

Memory-aware GGUF inference runtime with OpenAI-compatible serving

AI watermark removal built for Apple Silicon. SynthID-class pixel regeneration, visible vendor labels and provenance metadata, with every pi

The Kimi K3 and DeepSeek-V4-Flash-0731 architectures, reimplemented from scratch in pure Rust (zero dependencies) and verified 1:1 against t

🧠 One LLM, split across a Mac (Apple MPS) + a Windows PC (NVIDIA CUDA) — heterogeneous pipeline inference over a framework-neutral wire, bit

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen

Transformer inference via unconventional compute: a single algebraic primitive, a tag-system scheduler, and Apple's Neural Engine through co

Run a coding model at 300+ tok/s on your Mac. Native Rust + Metal inference engine for LiquidAI LFM2.5-24B-A2B (MLX 4-bit) on Apple Silicon.

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

Model Dispatch and Memory Allocator: an LLM inference engine for the hardware you own. Custom CPU, ROCm, Vulkan, CUDA, Metal and Level Zero

Native macOS app for local AI image generation and editing on Apple silicon: Z-Image Turbo, Qwen-Image-2.1, Krea 2 Turbo, Anima Turbo and MO

Local web UI for h3.c — native Metal video/audio generation with MiniMax-H3 on Apple Silicon, no PyTorch or MLX required.

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddings

Local speech-to-text at GPU speed: Parakeet, GigaAM and Whisper in pure Rust, as a library and an ollama-like CLI

Open-source native macOS terminal, up to 4× faster than Ghostty, with workspaces, notifications when Claude Code and other agents need you,

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.