Havuz

High performance inference engine

Foundational ternary-model inference & training library — CUDA + CPU, BitNet/TQ ternary kernels

AI inference library for mobile devices

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig

Native runtime, integrations, and reproducible packaging for BTL-3 and BTL-3 Compact

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in

A TUI around llama.cpp for running, managing, and benchmarking local GGUF models and launching the pi coding agent against your local server

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Use any AI model, locally or in the cloud, while tracking on-device token usage and collecting token telemetry data.

A web-based memory usage and performance calculator for Huggingface GGUF models

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

Published: 2026-10-18

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

neural networks in pure C

Multi-model orchestration engine. Combine any LLM providers (Gemini, OpenAI, Anthropic, Groq, local GGUF) into a single Ollama-compatible AP

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

High-performance, high-precision, locally-fast-deployed Jev-class medical decision service on national medical exams — four readings, one ro

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

DeepSeek V4 Flash 284B on AMD Strix Halo (gfx1151) — up to 32 tok/s decode & ~250 tok/s prefill via ROCmFPX, DSpark & ROCm 7.2

Evidence-backed structural validation of Kimi K3 UD-IQ1_M and UD-Q4_K_XL split GGUF releases using OMIV.

Speech-first inference runtime in Rust — TTS / ASR / speech-to-speech / VC / speaker ID / VAD. An ONNX Runtime alternative that loads GGUF &