Havuz

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

ENGRAFT: write facts into an LLM's n-gram (Engram) memory table as a removable overlay: 100 facts in a 9 MB file, no weights touched, verifi

llama.cpp performance patches developed and measured on Tesla P100 (GP100, sm_60). Most are not Pascal-specific.

Android keyboard with local AI (Ollama, Whisper, MCP) or cloud (Gemini, Groq, OpenAI)

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harnes

llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.

The models on your disk, put to work. A private desktop app for open models — chat, a coding agent built for small models, text-to-image, te

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Specul

Airgapped closed-corpus QA loop: a self-hosted Qwen3.6 agent explores a .zip dataroom under a token budget with local tools

AI commit messages from your local LLM - Ollama / llama-cpp backed, offline, Conventional Commits, with review + changelog modes.

Covert Coder — a local-first sovereign AI development environment for governed models, workflows, execution, evidence, and verification.

A personal AI agent to help you think, create, and get things done. Powered by open models, on your computer or in the cloud.

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Community benchmark database for running LLMs on Apple Silicon Macs

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

Lightweight proxy for LLM

Free, open-source AI coding agent for terminal and browser. A Claude Code / Cursor alternative that runs offline with Ollama or with your ow

为 JAV 日语视频生成日文、中文或双语 SRT 字幕。支持本地音视频识别、时间轴对齐及 API 或本地模型翻译；网页界面操作，视频与音频留在本机。适用于 Windows + NVIDIA 显卡。

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducib

Private, local-first AI meeting assistant for transcription, speaker diarization and meeting notes. Open-source, self-hosted alternative to

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen

One GPU. Full LLM workflow. Real benchmarks. No cloud required.

Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

One AI workspace for your models, agents, tools and computer.

Single-HTML monitoring dashboard for llama.cpp — per-slot sampling, plain-language tuning hints, router-mode aware, with built-in chat. Zero

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

A TUI around llama.cpp for running, managing, and benchmarking local GGUF models and launching the pi coding agent against your local server

Can't find subtitles? Generate SRT/ASS subtitles from any video with local AI and auto-translate them — no internet, free. macOS & Windows.

Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simula

Dive is a secure, local-first desktop client for LllamaCCP, LmStudio, Ollama, Pi agentic AI and Cloud. Features an ultra-compact offline RAG

A web-based memory usage and performance calculator for Huggingface GGUF models

LM Local is an AI coding assistant for Visual Studio 2022/2026 with agentic capabilities and subagents

The operations layer for your local LLM stack

Free, open-source Mac dictation built for accessibility. Press a key, speak, and it types in any app. Whisper and AI cleanup run fully on yo

Self-hosted LLM/RAG stack in one command — AMD Strix Halo / x86_64 (ROCm/Vulkan, Docker Compose)

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

Self-hosted, browser-based AI dungeon RPG, fully local on an AMD Strix Halo APU: narrator + per-character agents on one local LLM (Gemma 4 2

[Archived] AAOSP, predecessor to JibarOS. Early experiment in OS-level on-device AI (MCP + local LLM on Android 15). See JibarOS for the act

Introducing Muse Glimmer: open-weight 30B agentic multimodal model that runs on your device (Meta). Interactive local agent lab + guide. Apa

Auto-tuning de modelos locais: TPS e contexto sob limite de PPL, com busca finita, protecoes de recursos e evidencias em SQLite.

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

Summarize, explain, fact-check, or translate any text, URL, or file. No GPU. No cloud. One command

Run the LLM Swarm Router on machines to distribute Local Ai to the Swarm - More Machines - MORE SPEED

Drop-in ggml backend for Rockchip NPUs: offloads llama.cpp/whisper.cpp prefill to the RK3588 NPU

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

Typed choices and probability distributions from text and images with llama.cpp, Ollama, and OpenRouter