Havuz

Qwen3.8-27B on RTX 5090 Laptop 24GB — TWIN-TURBO NVFP4 (native llama.cpp) vs Coder390 Q3 LynnStyle (.ninfer + NInfer engine in WSL2/Docker):

Unified native AI runtime for Java 17+, Java 22+ (Panama FFM), Android & C++. Run GGUF LLMs, Vision, Video, Whisper ASR, TTS & Embeddings in

MCP server that delegates Claude Code subagents to local models (LiteLLM/llama.cpp/Ollama), DeepSeek, AWS Bedrock, or any OpenAI/Anthropic-c

A memory system for AI agents modeled on human memory: episodic, affective, reconstructive.

OnDevice Local AI Studio — run LLMs on iPhone offline. SwiftUI workbench with MLX, llama.cpp, whisper.cpp, RAG, and authenticated OpenAI/Ant

Find anything on your PC by what's in it: local, private semantic search for Windows. Photos, screenshots, PDFs, documents, recordings and v

Boot your PC straight into an LLM. Rust, UEFI-resident, no operating system underneath.

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Fully local voice AI for iOS

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

Hermes Agent (NousResearch) sandboxed by NVIDIA OpenShell — hardware-enforced network/filesystem/syscall policy, full memory + gateway stack

Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the

NeveAI é uma plataforma de IA local privacy-first, desenvolvida para oferecer uma experiência de alta performance na execução de LLMs, reduz

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.

Community benchmark database for running LLMs on Apple Silicon Macs

Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

Native AI assistant for macOS, built with SwiftUI and llama.cpp. Run local LLMs offline, work with PDFs, and create custom AI assistants. Op

One AI workspace for your models, agents, tools and computer.

Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simula

Dive is a secure, local-first desktop client for LllamaCCP, LmStudio, Ollama, Pi agentic AI and Cloud. Features an ultra-compact offline RAG

LM Local is an AI coding assistant for Visual Studio 2022/2026 with agentic capabilities and subagents

Free, open-source Mac dictation built for accessibility. Press a key, speak, and it types in any app. Whisper and AI cleanup run fully on yo

Self-hosted LLM/RAG stack in one command — AMD Strix Halo / x86_64 (ROCm/Vulkan, Docker Compose)

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

[Archived] AAOSP, predecessor to JibarOS. Early experiment in OS-level on-device AI (MCP + local LLM on Android 15). See JibarOS for the act

Drop-in ggml backend for Rockchip NPUs: offloads llama.cpp/whisper.cpp prefill to the RK3588 NPU

Fit-to-Size Intelligent Tensor Quantization for GGUF - deterministic, size-exact mixed quantization on llama.cpp presets. Turn any byte budg

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

A NPC AI Framework , based on character personalities, can be used for game NPC (extendable for Godot / Unity)

Which local LLMs can your computer run? Hardware-aware CLI, TUI and GUI: yes/slow/no verdicts and tok/s from your GPU VRAM and RAM, then ver

Run a team of AI agents on your own computer. Your files, browser sessions and passwords stay local. Works with Claude Code, Codex, hosted o

一键本地 AI 工作站：模型市场（多源测速+断点续传）、llama.cpp+MNN 双引擎、多芯片检测（NVIDIA/AMD ROCm/华为昇腾）、LLM/TTS/图像/视频/3D/超分、Agent 助手；Electron + 原生 JS + Python sidecar，数据默

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

Local-first semantic search engine for your files — hybrid BM25 + vector + HyDE search over docs, PDFs, images, audio and video. 100% offlin

BANTAM FACTORY — Your model. Your hardware. A coding factory that can improve itself.

面向内存受限端侧设备的 MoE 大模型推理运行时优化系统

Multi-GPU prefill acceleration for llama.cpp

方寸智匣 LocalBrain：本地大模型、文件与媒体工作台。Windows/Linux 2.2.8，Apple Silicon Mac 2.2.7；按平台维护运行环境，支持本地工具与 MCP 接入。

Qwen 3.8 27B on a $1,400 Strix Halo: frozen optimal config, 262k needle-exact retrieval, vision fixed, delegation-card certified

Turn any document or a whole zip into an interactive knowledge graph, using a self-hosted Qwen3.6-35B-A3B-MTP on a single NVIDIA L4

A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share

Local LLM Server Manager + LlaMA.cpp + Chat

Top AI Calorie Tracker GitHub 2026

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accu

Selfhost modern LLM stacks. Run the whole fleet from your terminal

OpenVitamin is a local-first AI execution platform that unifies Agents, Workflows, and multi-model inference into a single programmable syst

Airgapped closed-corpus QA loop: a self-hosted Qwen3.6 agent explores a .zip dataroom under a token budget with local tools