Havuz

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Local inference for diffusion language models (LLaDA, Dream), written in Mojo

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cp

Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml: Qwen3-TTS, Irodori-TTS, Qwen3-ASR,

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

Pure-Rust MoE inference engine for Qwen3.5/3.6 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp pref

Screen Translator, Android ekranındaki yazıları OCR ile tanıyıp çeviren ve sonucu ekranın üzerine yerleştiren bir uygulamadır.

Neden yükseliyor · 97 günde 906 yıldız alması, özellikle oyun ve manga çevirisi gibi belirgin bir kullanım alanına hitap ettiğine işaret ediyor. Ancak son 28 günde 2 commit ve 1 farklı yazar bilgisi, ilginin sürdürülebilir geliştirme faaliyetiyle desteklendiğine dair sınırlı kanıt sunuyor.

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https

Bilingual English and Mandarin text to speech in C++ and GGUF. Voice design, cloning, direction and experimental voice conversion. Vulkan ba

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine f

JevStyle: local decision models for AI agents. Classify, route and score with a probability per option. systemone-compatible local server, M

🐋 Run 284B DeepSeek-V4-Flash (<59GiB RAM, 44.4 tok/s pp, 9.1-14.7 tok/s) & 30B MoEs (25-111 tok/s) on a pure 64GB CPU at 256K ctx (0 GPUs).

Atomic Agent, yerel veya cloud modellerle çalışan ve browser, dosya, onaylı shell komutları ile MCP araçlarını kullanabilen bir AI agent uygulamasıdır.

Neden yükseliyor · 159 günlük bir repoda 2.505 yıldız ve son 28 günde 648 commit dikkat çekiyor; yerel modellerle çalışan kapsamlı bir agent vaadi ve README'deki GAIA L1 karşılaştırması ilgiyi artırmış olabilir. Ancak commitlerin yalnızca 2 farklı yazardan gelmesi aktivitenin geniş bir katkıcı tabanına yayıldığını göstermiyor; yükselişin kesin nedeni bilinmiyor.

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimiz

Local speech-to-text at GPU speed: Parakeet, GigaAM and Whisper in pure Rust, as a library and an ollama-like CLI

The most advanced, fully offline client-side AI suite on Android today.

llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s

A dependency-free C inference engine for Liquid AI's LFM2 / LFM2.5 models. Terminal chat, an OpenAI-compatible server, a web agent and a ter

Private, offline AI assistant for Android and Windows: run local LLMs (Gemma 4, Qwen, any GGUF) on your phone or PC with llama.cpp, CUDA and

Local-first AI chat client combining SillyTavern-style character cards, LM Studio-style GGUF model management, Ollama/OpenAI-compatible APIs

A 4B decision model: calibrated probabilities for yes/no, choice and score questions over long inputs.

High-performance full-text search across PDFs, Office documents, ZIP archives, email, media metadata and GGUF/LLM model files. ripgrep-compa

Open-source voice typing for Windows and macOS — a Wispr Flow / Superwhisper alternative. Press a shortcut, speak, and AI-polished text land

译声实时翻译（白色新版）— 免费的 Windows 实时日语字幕与翻译工具，支持系统声音、本地 GGUF / Ollama / OpenAI 兼容 API 翻译。

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

User friendly GUI for configuring and launching llama.cpp

A 6.87 GiB GGUF build of Qwen 3.8 27B — full 27B quality on 8 GB RAM. 96.7% HumanEval, 92.9% MMLU retention, no distillation.

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

High performance inference engine

Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).

Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.

Local LLM inference on Xbox Series S in Developer Mode: llama.cpp (GGUF) + ONNX Runtime GenAI, an OpenAI-compatible LAN endpoint (off by def

Clonación de voz y texto a voz 100% local con Qwen3-TTS y llama.cpp. Funciona en CPU y GPU, 10 idiomas, interfaz web en español.

Qwen3.8-Flash-Next 177B MoE (IQ3_S 3.44bpw) on RTX 5090 Laptop 24GB: full-model quality @ 97.7 tok/s · dual-lane 256K full-context unlocked

Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs

Local AI desktop app: a Chromium browser with a GPU-aware llama.cpp runtime, OpenAI-compatible API, AI chat and agents, Stable Diffusion ima

Electron + Next.js desktop AI assistant that runs GGUF models locally using llama.cpp. Designed for offline use, portability, and zero-insta

Native macOS app for local AI image generation and editing on Apple silicon: Z-Image Turbo, Qwen-Image-2.1, Krea 2 Turbo, Anima Turbo and MO

Look inside any local LLM (GGUF from Ollama, LM Studio, Hugging Face, Jan, llama.cpp): layers, heads, vocabulary, chat template, weights. Ex

Evidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results.

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verifi

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Agentic Runtime

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

Fit-to-Size Intelligent Tensor Quantization for GGUF - deterministic, size-exact mixed quantization on llama.cpp presets. Turn any byte budg

🔥 Run large open language models on your phone. A C++20 GGUF inference engine that streams models larger than RAM from flash, paces itself b