Havuz

Dive is a secure, local-first desktop client for LllamaCCP, LmStudio, Ollama, Pi agentic AI and Cloud. Features an ultra-compact offline RAG

A web-based memory usage and performance calculator for Huggingface GGUF models

Free, open-source Mac dictation built for accessibility. Press a key, speak, and it types in any app. Whisper and AI cleanup run fully on yo

Self-hosted LLM/RAG stack in one command — AMD Strix Halo / x86_64 (ROCm/Vulkan, Docker Compose)

Summarize, explain, fact-check, or translate any text, URL, or file. No GPU. No cloud. One command

Run the LLM Swarm Router on machines to distribute Local Ai to the Swarm - More Machines - MORE SPEED

Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX cards (tuned on 3080 10gb). A measured, tuned and benchmarked fork of Str

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

GamePause for Local AI: a Windows tray app that pauses LM Studio, Ollama and other local AI servers while you game, and restores them afterw

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

Typed choices and probability distributions from text and images with llama.cpp, Ollama, and OpenRouter

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

CubicLM is a unified cross-platform AI workspace with local GPU acceleration (GGUF, LiteRT-LM). Supports on-device image generation, 23+ clo

Copper does not rust and is essential for modern infrastructure, written in rust

Fine-tune Qwen2.5-Coder-7B for free (QLoRA on Kaggle T4) and run a local coding agent that runs your code instead of guessing — 8 GB Mac, ll

Which local LLMs can your computer run? Hardware-aware CLI, TUI and GUI: yes/slow/no verdicts and tok/s from your GPU VRAM and RAM, then ver

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Dock

Local agentic AI runtime optimized for CPU-only consumer hardware, tool use and efficient local inference.

Run a team of AI agents on your own computer. Your files, browser sessions and passwords stay local. Works with Claude Code, Codex, hosted o

一键本地 AI 工作站：模型市场（多源测速+断点续传）、llama.cpp+MNN 双引擎、多芯片检测（NVIDIA/AMD ROCm/华为昇腾）、LLM/TTS/图像/视频/3D/超分、Agent 助手；Electron + 原生 JS + Python sidecar，数据默

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

Local-first semantic search engine for your files — hybrid BM25 + vector + HyDE search over docs, PDFs, images, audio and video. 100% offlin

BANTAM FACTORY — Your model. Your hardware. A coding factory that can improve itself.

Local AI model manager

AI Infra 推理系统学习资料：从深度学习算子到 vLLM 服务端部署

Measures which GGUF quant of a model to run on your GPU: real speed and quality loss (KL divergence), not estimates.

SDK for building on-device AI applications on Arm with reusable building blocks across inference runtimes.

Benchmark llama.cpp on your own GPU/CPU boxes and compare against everyone else's — sweep runner, an sha256-keyed index of ~4M GGUF files, a

Multi-GPU prefill acceleration for llama.cpp

An agent that builds agents — closed-loop AI system that interviews, designs, tests, and ships standalone Python agents.

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.

Portable offline AI assistant powered by llama.cpp and GGUF models. Run a private local chatbot directly from a USB drive on Windows with no

USB-portable offline AI assistant. AES-256 encrypted history, Turkish-tuned LLM (Trendyol), RAG over PDF/DOCX. KVKK uyumlu, %100 offline.

Qwen 3.8 27B on a $1,400 Strix Halo: frozen optimal config, 262k needle-exact retrieval, vision fixed, delegation-card certified

single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

MCP server that delegates Claude Code agent work to local models (llama.cpp / vLLM) on your own hardware

MoE expert compression toolkit — prune, merge, and forge specialist models for local inference.

OraType (formerly Typelite). Just say it. Free, open-source voice typing for macOS: speak in any app and get clean text. Runs on your comput

Ternary decision scoring with Jevfire-style inference, typed APIs, and reproducible Persian evaluation.

A production-oriented LLM engineering platform with OpenAI-compatible serving, streaming, observability, evaluation, reproducible experiment

Evidence-backed structural validation of Kimi K3 UD-IQ1_M and UD-Q4_K_XL split GGUF releases using OMIV.

High-performance, high-precision, locally-fast-deployed Jev-class medical decision service on national medical exams — four readings, one ro

Arabic-first generative speech recognition — Audar-ASR-V1 (Flash + Turbo). #1 on the Open Universal Arabic ASR Leaderboard. Model cards, ben

Local LLM Server Manager + LlaMA.cpp + Chat

Top AI Calorie Tracker GitHub 2026

Lightweight agent that pulls and runs LLM benchmarks from Protorikis Bench

Self-hosted AI workspace where chat becomes visual workflows, multi-agent operations, and reviewable automations. Local memory; local or clo

Pure-Rust MoE inference engine for Qwen3.5/3.6 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp pref