Havuz

Typed decisions from unstructured state: one forward pass, zero generated text. Local, deterministic, Jev-compatible. Not affiliated with ty

High performance inference engine

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

Published: 2026-10-18

AI inference library for mobile devices

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

Portable offline AI assistant powered by llama.cpp and GGUF models. Run a private local chatbot directly from a USB drive on Windows with no

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig

A TUI around llama.cpp for running, managing, and benchmarking local GGUF models and launching the pi coding agent against your local server

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Use any AI model, locally or in the cloud, while tracking on-device token usage and collecting token telemetry data.

A web-based memory usage and performance calculator for Huggingface GGUF models

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

Rivo Agent is a private, local-first mobile AI assistant that runs compact GGUF language models directly on your phone. It lets users downlo

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

neural networks in pure C

High-performance, high-precision, locally-fast-deployed Jev-class medical decision service on national medical exams — four readings, one ro

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

RenderCaption - Offline AI Speech to Text & Subtitles Generator.

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

Evidence-backed structural validation of Kimi K3 UD-IQ1_M and UD-Q4_K_XL split GGUF releases using OMIV.

Run local LLMs entirely in your browser, with no server, no install and no traces. A private, single-file AI chat UI and AGENT that also wor

DeepSeek V4 Flash 284B on AMD Strix Halo (gfx1151) — up to 32 tok/s decode & ~250 tok/s prefill via ROCmFPX, DSpark & ROCm 7.2

Speech-first inference runtime in Rust — TTS / ASR / speech-to-speech / VC / speaker ID / VAD. An ONNX Runtime alternative that loads GGUF &

Local AI model manager