Havuz

Semantic search in one SQLite file. No model, no server at query time.

Fit-to-Size Intelligent Tensor Quantization for GGUF - deterministic, size-exact mixed quantization on llama.cpp presets. Turn any byte budg

Run a coding model at 300+ tok/s on your Mac. Native Rust + Metal inference engine for LiquidAI LFM2.5-24B-A2B (MLX 4-bit) on Apple Silicon.

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

Rivo Agent is a private, local-first mobile AI assistant that runs compact GGUF language models directly on your phone. It lets users downlo

CubicLM is a unified cross-platform AI workspace with local GPU acceleration (GGUF, LiteRT-LM). Supports on-device image generation, 23+ clo

译声实时翻译（白色新版）— 免费的 Windows 实时日语字幕与翻译工具，支持系统声音、本地 GGUF / Ollama / OpenAI 兼容 API 翻译。

A NPC AI Framework , based on character personalities, can be used for game NPC (extendable for Godot / Unity)

Copper does not rust and is essential for modern infrastructure, written in rust

Qwen3.8-27B on RTX 5090 Laptop 24GB — TWIN-TURBO NVFP4 (native llama.cpp) vs Coder390 Q3 LynnStyle (.ninfer + NInfer engine in WSL2/Docker):

Fine-tune Qwen2.5-Coder-7B for free (QLoRA on Kaggle T4) and run a local coding agent that runs your code instead of guessing — 8 GB Mac, ll

Which local LLMs can your computer run? Hardware-aware CLI, TUI and GUI: yes/slow/no verdicts and tok/s from your GPU VRAM and RAM, then ver

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

A memory system for AI agents modeled on human memory: episodic, affective, reconstructive.

Local AI video dubbing studio with voice cloning, speech translation, speaker diarization and audio separation. Edit, preview and export dub

Local agentic AI runtime optimized for CPU-only consumer hardware, tool use and efficient local inference.

Run a team of AI agents on your own computer. Your files, browser sessions and passwords stay local. Works with Claude Code, Codex, hosted o

一键本地 AI 工作站：模型市场（多源测速+断点续传）、llama.cpp+MNN 双引擎、多芯片检测（NVIDIA/AMD ROCm/华为昇腾）、LLM/TTS/图像/视频/3D/超分、Agent 助手；Electron + 原生 JS + Python sidecar，数据默

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

A dashboard for a home cluster of NVIDIA GB10 (DGX Spark / ASUS GX10) inference servers: GPU and system health, plus live per-model metrics

Local-first semantic search engine for your files — hybrid BM25 + vector + HyDE search over docs, PDFs, images, audio and video. 100% offlin

BANTAM FACTORY — Your model. Your hardware. A coding factory that can improve itself.

Local AI model manager

面向内存受限端侧设备的 MoE 大模型推理运行时优化系统

AI Infra 推理系统学习资料：从深度学习算子到 vLLM 服务端部署

Measures which GGUF quant of a model to run on your GPU: real speed and quality loss (KL divergence), not estimates.

SDK for building on-device AI applications on Arm with reusable building blocks across inference runtimes.

AI-assisted rescue boot USB — a Ventoy multi-boot drive on Ubuntu 24.04 with offline LLM inference (Ollama + llama.cpp), pre-installed agent

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Lightweight agent that pulls and runs LLM benchmarks from Protorikis Bench

Multi-GPU prefill acceleration for llama.cpp

Find anything on your PC by what's in it: local, private semantic search for Windows. Photos, screenshots, PDFs, documents, recordings and v

An agent that builds agents — closed-loop AI system that interviews, designs, tests, and ships standalone Python agents.

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.

Portable offline AI assistant powered by llama.cpp and GGUF models. Run a private local chatbot directly from a USB drive on Windows with no

Proof of concept: turn old Android phones into an LLM inference cluster. ADB zero-touch provisioning, llama.cpp on each phone, OpenAI-compat

USB-portable offline AI assistant. AES-256 encrypted history, Turkish-tuned LLM (Trendyol), RAG over PDF/DOCX. KVKK uyumlu, %100 offline.

方寸智匣 LocalBrain：本地大模型、文件与媒体工作台。Windows/Linux 2.2.8，Apple Silicon Mac 2.2.7；按平台维护运行环境，支持本地工具与 MCP 接入。

Qwen 3.8 27B on a $1,400 Strix Halo: frozen optimal config, 262k needle-exact retrieval, vision fixed, delegation-card certified

single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

MCP server that delegates Claude Code agent work to local models (llama.cpp / vLLM) on your own hardware

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

OraType (formerly Typelite). Just say it. Free, open-source voice typing for macOS: speak in any app and get clean text. Runs on your comput

Ternary decision scoring with Jevfire-style inference, typed APIs, and reproducible Persian evaluation.

A production-oriented LLM engineering platform with OpenAI-compatible serving, streaming, observability, evaluation, reproducible experiment

Evidence-backed structural validation of Kimi K3 UD-IQ1_M and UD-Q4_K_XL split GGUF releases using OMIV.

High-performance, high-precision, locally-fast-deployed Jev-class medical decision service on national medical exams — four readings, one ro

Pure-Rust MoE inference engine for Qwen3.5/3.6 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp pref