Havuz

ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenC

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine f

AnyJev, open LLM’lerden typed kararlar ve eşiklenebilir olasılıklar üretmek için calibration ve karar katmanları sunan bir Python kütüphanesidir.

Neden yükseliyor · Yedi günde 802 yıldız almış; dikkat çekmesinde etiket gerektirmeyen ilk seviye, az etiketle kalibrasyon iddiası ve modeli fine-tune etmeden karar endpoint’i sunma yaklaşımı etkili görünüyor. Bu ilgi benimseme kanıtı değil; verilen sonuçlar README’deki sınırlı benchmark’a dayanıyor.

Spark-X2.5, yerel çalıştırma ve agent iş akışları için tasarlanmış 1.7B ve 4B boyutlarında açık dil modelleri serisidir.

Neden yükseliyor · Yeni model serisi; uzun context, küçük model boyutu ve agent/coding yeteneklerini aynı anda öne çıkarıyor. README'de 34 günlük yaş, 386 yıldız ve son 28 günde 9 commit ile 2 farklı yazar bilgisi var; bunlar erken ilgi ve güncel çalışma gösteriyor, ancak model performansını bağımsız olarak kanıtlamıyor.

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how

Halo is an open-source framework built by White Circle for training large language and multimodal models

本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块；vLLM / llama.cpp / Transformers 多引擎本地推理，显存智能调度、舱壁隔离自愈、仅供学习使用，欢迎交流指正。（最新进度（除插件系统）受到协议限制，

ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuous

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-awa

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

dgx-spark-qwen38 is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Ace

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memo

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Python package for LLM compression

Your AI intranet: network the computers you already own for inference and training.

An open-source inference engine for AI functions in SQL

Think only when the tool call looks unsure: confidence-gated reasoning for small local models. Code, BFCL results and paper.

📚 An educational, step-by-step implementation of LLM KV cache reuse — learn prefix caching, chunk keys, storage & eviction, from pure Python

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM i

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

Open-source local implementation of MiniMax H3 Context-IR. Turn simple prompts into structured, validated H3 video briefs.

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decodi

Open, calibrated, single-pass typed-decision models (4B & 27B) for finance and trading

Hemmingway-1: a 27B model that writes the way a person writes. Open weights, free for non-commercial use (CC BY-NC 4.0); commercial use by a

Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVID

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

Decisions, not essays: a choice, a yes or no, or a score with a calibrated probability. Open weights, Jev-compatible API, adaptive thinking.

GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec d

Local diagnostic CLI for NVIDIA DGX Spark (GB10). Detects power caps, UMA pressure, thermal risk, CUDA 13/SM_121 wheel mismatches, Docker is

The open-core AI workbench — notebooks, agents, RAG, voice, and images across any model: OpenAI, Anthropic, Google, xAI, or local via Ollama

Disaggregated quantization official codebase

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Specul

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A

one-year LLM serving trace from Chutes

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hos

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind

Turn documents into structured JSON with local-first document AI. Run 100% locally by default, with API, CLI, and Web UI.

Network-faithful simulation of LLM serving and training deployments