Havuz

The open-core AI workbench — notebooks, agents, RAG, voice, and images across any model: OpenAI, Anthropic, Google, xAI, or local via Ollama

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-awa

Your AI intranet: network the computers you already own for inference and training.

An open-source inference engine for AI functions in SQL

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Python package for LLM compression

Turn documents into structured JSON with local-first document AI. Run 100% locally by default, with API, CLI, and Web UI.

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memo

Halo is an open-source framework built by White Circle for training large language and multimodal models

AnyJev, open LLM’lerden typed kararlar ve eşiklenebilir olasılıklar üretmek için calibration ve karar katmanları sunan bir Python kütüphanesidir.

Neden yükseliyor · Yedi günde 802 yıldız almış; dikkat çekmesinde etiket gerektirmeyen ilk seviye, az etiketle kalibrasyon iddiası ve modeli fine-tune etmeden karar endpoint’i sunma yaklaşımı etkili görünüyor. Bu ilgi benimseme kanıtı değil; verilen sonuçlar README’deki sınırlı benchmark’a dayanıyor.

Spark-X2.5, yerel çalıştırma ve agent iş akışları için tasarlanmış 1.7B ve 4B boyutlarında açık dil modelleri serisidir.

Neden yükseliyor · Yeni model serisi; uzun context, küçük model boyutu ve agent/coding yeteneklerini aynı anda öne çıkarıyor. README'de 34 günlük yaş, 386 yıldız ve son 28 günde 9 commit ile 2 farklı yazar bilgisi var; bunlar erken ilgi ve güncel çalışma gösteriyor, ancak model performansını bağımsız olarak kanıtlamıyor.

本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Open-source local implementation of MiniMax H3 Context-IR. Turn simple prompts into structured, validated H3 video briefs.

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind

Network-faithful simulation of LLM serving and training deployments

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuous

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hos

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — de

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression wit

Open-source, self-hostable social listening.

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM i

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference speed

Self-hosted Chess.com alternative: Game Review for your Chess.com & Lichess games, play vs Stockfish or a friend, lessons + opening trainer,

The open-source AI platform for enterprises that can't send data to the cloud. OpenAI-compatible API, full management dashboard, zero data e

Production-ready, bleeding-edge, upstream-first vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a), verified against real models on

Android keyboard with local AI (Ollama, Whisper, MCP) or cloud (Gemini, Groq, OpenAI)

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harnes

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

Legacy: self-hosted LLM/RAG stack installer for DGX Spark (arm64). The work continued as AGmind Systems Lab — measured local-LLM performance

Plug-and-play, dependency-free web command center for DGX Spark fleets (and any GPU boxes): live per-node GPU/CPU/mem/temps/power, optional

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Hemmingway-1: a 27B model that writes the way a person writes. Open weights, free for non-commercial use (CC BY-NC 4.0); commercial use by a

Local-first CLI for benchmarking LLMs on real hardware — quality, speed, reliability, and a real multi-turn agent loop.

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

Lightweight proxy for LLM