Havuz

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-awa

Your AI intranet: network the computers you already own for inference and training.

An open-source inference engine for AI functions in SQL

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuous

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine f

Python package for LLM compression

AnyJev, open LLM’lerden typed kararlar ve eşiklenebilir olasılıklar üretmek için calibration ve karar katmanları sunan bir Python kütüphanesidir.

Neden yükseliyor · Yedi günde 802 yıldız almış; dikkat çekmesinde etiket gerektirmeyen ilk seviye, az etiketle kalibrasyon iddiası ve modeli fine-tune etmeden karar endpoint’i sunma yaklaşımı etkili görünüyor. Bu ilgi benimseme kanıtı değil; verilen sonuçlar README’deki sınırlı benchmark’a dayanıyor.

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memo

ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenC

Halo is an open-source framework built by White Circle for training large language and multimodal models

Spark-X2.5, yerel çalıştırma ve agent iş akışları için tasarlanmış 1.7B ve 4B boyutlarında açık dil modelleri serisidir.

Neden yükseliyor · Yeni model serisi; uzun context, küçük model boyutu ve agent/coding yeteneklerini aynı anda öne çıkarıyor. README'de 34 günlük yaş, 386 yıldız ve son 28 günde 9 commit ile 2 farklı yazar bilgisi var; bunlar erken ilgi ve güncel çalışma gösteriyor, ancak model performansını bağımsız olarak kanıtlamıyor.

本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Open-source local implementation of MiniMax H3 Context-IR. Turn simple prompts into structured, validated H3 video briefs.

dgx-spark-qwen38 is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Ace

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

Local diagnostic CLI for NVIDIA DGX Spark (GB10). Detects power caps, UMA pressure, thermal risk, CUDA 13/SM_121 wheel mismatches, Docker is

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hos

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — de

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

The open-core AI workbench — notebooks, agents, RAG, voice, and images across any model: OpenAI, Anthropic, Google, xAI, or local via Ollama

Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression wit

Open-source, self-hostable social listening.

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM i

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference speed

Turn documents into structured JSON with local-first document AI. Run 100% locally by default, with API, CLI, and Web UI.

GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec d

The open-source AI platform for enterprises that can't send data to the cloud. OpenAI-compatible API, full management dashboard, zero data e

Production-ready, bleeding-edge, upstream-first vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a), verified against real models on

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Self-hosted Chess.com alternative: Game Review for your Chess.com & Lichess games, play vs Stockfish or a friend, lessons + opening trainer,

Android keyboard with local AI (Ollama, Whisper, MCP) or cloud (Gemini, Groq, OpenAI)

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harnes

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Specul

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding