Havuz

The models on your disk, put to work. A private desktop app for open models — chat, a coding agent built for small models, text-to-image, te

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

Pools consumer laptops into one local runtime for Meta Llama models. Measures the free VRAM each machine reports, splits the model across CU

CPU-optimized LLM inference engine (C)

🏆Winning Project | ModelGate is a contract-aware AI control plane that ingests customer contracts, extracts SLA/privacy/routing constraints,

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Agentic Runtime

Hemmingway-1: a 27B model that writes the way a person writes. Open weights, free for non-commercial use (CC BY-NC 4.0); commercial use by a

一套功能完整、轻量稳定的 ComfyUI 实用节点集。初衷是为了平替一些断更插件的功能，会持续更新一些新开发的节点插件。有好的创意可以发Issues，我会力所能及想办法做出来。全中文界面，原生控件，不依赖第三方库。

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

Universal local LLM runtime. Drop-in Ollama replacement with HuggingFace integration, pipelines, structured output. Free & open source.

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducib

Crow — a local AI agent, not a chat box: 30 tools, MCP, memory, skills, goals, subagents, vision, and image generation and editing with Qwen

Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVID

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and be

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Ra

One GPU. Full LLM workflow. Real benchmarks. No cloud required.

AI inference library for mobile devices

llama.cpp speculative decoding measured on one RTX 3090, Qwen3.6-35B-A3B UD-Q4_K_XL, commit 3737e4137. Published figures are re-derived from

Impulse by Velocity: run AI models on your own computer, on Windows and Linux. Load any GGUF model and chat with web search, files, charts a

Local AI for your whole house: chat, images and voice on your own GPU. Computes each model's flags, fits them to your VRAM, and hot-swaps th

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flas

Native AI assistant for macOS, built with SwiftUI and llama.cpp. Run local LLMs offline, work with PDFs, and create custom AI assistants. Op

A TUI around llama.cpp for running, managing, and benchmarking local GGUF models and launching the pi coding agent against your local server

Piccolo: 4B, 9B and 27B decision models for agent tool review, context retention and data workflows, with an interactive Arena.

A web-based memory usage and performance calculator for Huggingface GGUF models

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, b

neural networks in pure C

A dependency-free C inference engine for Liquid AI's LFM2 / LFM2.5 models. Terminal chat, an OpenAI-compatible server, a web agent and a ter

Local AI engine in Rust: run GGUF and ONNX models for chat, vision, speech and embeddings on your own GPU, RAM and CPU, with automatic place

Memory-aware GGUF inference runtime with OpenAI-compatible serving

Qwen3.8-Flash-Next 177B MoE (IQ3_S 3.44bpw) on RTX 5090 Laptop 24GB: full-model quality @ 97.7 tok/s · dual-lane 256K full-context unlocked

Foundational ternary-model inference & training library — CUDA + CPU, BitNet/TQ ternary kernels

译声实时翻译（白色新版）— 免费的 Windows 实时日语字幕与翻译工具，支持系统声音、本地 GGUF / Ollama / OpenAI 兼容 API 翻译。

Local AI workstation: chat by voice or text, transcribe, generate images and video (with sound on Apple Silicon), benchmark and serve open-w

Winning Qwen and DeepSeek setups for AMD Strix Halo: pinned Ansible recipes and matched benchmarks on a 128 GiB Ryzen AI Max+ 395 / Radeon 8

Locally fine-tuned Russian RAG models: document splitter, query expansion, retrieval embedder. Teacher distillation → GGUF → llama.cpp on AM

A supercharged terminal for Mac with built-in local AI — suggestions, error explanations, and /ask chat. No API key required.

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

Local LLMs that run lean, load instantly and keep getting smarter. A from-scratch runtime + model format in Zig: zero-copy mmap, CPU/CUDA/Vu

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

Fit-to-Size Intelligent Tensor Quantization for GGUF - deterministic, size-exact mixed quantization on llama.cpp presets. Turn any byte budg

Qwen3.8-27B on one laptop CPU: up to 2.52 token/s, 8 GB tested, no runtime accuracy loss. Native OpenAI-compatible API with function tools;

The best model under 200B at near 10 token/s exact batch throughput on a single laptop CPU, with an automatic 8 GB RAM path. Native C, no GP

Local speech-to-text at GPU speed: Parakeet, GigaAM and Whisper in pure Rust, as a library and an ollama-like CLI

Arabic-first expressive zero-shot speech synthesis — Audar-TTS-V1 (Flash + Turbo + Pro). Voice cloning, expression tags, benchmarks, technic

High-performance full-text search across PDFs, Office documents, ZIP archives, email, media metadata and GGUF/LLM model files. ripgrep-compa

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig