Havuz

A TUI around llama.cpp for running, managing, and benchmarking local GGUF models and launching the pi coding agent against your local server

A web-based memory usage and performance calculator for Huggingface GGUF models

Memory-aware GGUF inference runtime with OpenAI-compatible serving

Foundational ternary-model inference & training library — CUDA + CPU, BitNet/TQ ternary kernels

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

Local LLMs that run lean, load instantly and keep getting smarter. A from-scratch runtime + model format in Zig: zero-copy mmap, CPU/CUDA/Vu

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig

Rivo Agent is a private, local-first mobile AI assistant that runs compact GGUF language models directly on your phone. It lets users downlo

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Use any AI model, locally or in the cloud, while tracking on-device token usage and collecting token telemetry data.

RenderCaption - Offline AI Speech to Text & Subtitles Generator.

Local AI model manager

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.

Portable offline AI assistant powered by llama.cpp and GGUF models. Run a private local chatbot directly from a USB drive on Windows with no

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

Speech-first inference runtime in Rust — TTS / ASR / speech-to-speech / VC / speaker ID / VAD. An ONNX Runtime alternative that loads GGUF &

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

Local, offline text-to-speech and voice cloning on audio.cpp — multi-voice narration, voice design, sound/music generation, and a real-time

Run open AI models on your own computer — a friendly desktop app and local gateway for llama.cpp and more. Private, free, open source.

Ternary decision scoring with Jevfire-style inference, typed APIs, and reproducible Persian evaluation.

Evidence-backed structural validation of Kimi K3 UD-IQ1_M and UD-Q4_K_XL split GGUF releases using OMIV.

ComfyUI deployment kit for Qwen-Image-2.1 Uncensored (GGUF): ready-to-use t2i & image-edit workflows, ComfyUI-GGUF architecture patches, and

Typed decisions from unstructured state: one forward pass, zero generated text. Local, deterministic, Jev-compatible. Not affiliated with ty

Arabic-first generative speech recognition — Audar-ASR-V1 (Flash + Turbo). #1 on the Open Universal Arabic ASR Leaderboard. Model cards, ben

Zero-G Reverse Engineering Anything & Verified Safe Rust Reconstruction for Gemini 4 Argon + Google Antigravity 2.0 (4-Mount Recon Kits, Who

Fine-tune Qwen2.5-Coder-7B for free (QLoRA on Kaggle T4) and run a local coding agent that runs your code instead of guessing — 8 GB Mac, ll

Measures which GGUF quant of a model to run on your GPU: real speed and quality loss (KL divergence), not estimates.

Published: 2026-10-19

Published: 2026-10-18

Published: 2026-10-18

Multi-model orchestration engine. Combine any LLM providers (Gemini, OpenAI, Anthropic, Groq, local GGUF) into a single Ollama-compatible AP

High-performance, high-precision, locally-fast-deployed Jev-class medical decision service on national medical exams — four readings, one ro