Havuz

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine f

AnyJev, open LLM’lerden typed kararlar ve eşiklenebilir olasılıklar üretmek için calibration ve karar katmanları sunan bir Python kütüphanesidir.

Neden yükseliyor · Yedi günde 802 yıldız almış; dikkat çekmesinde etiket gerektirmeyen ilk seviye, az etiketle kalibrasyon iddiası ve modeli fine-tune etmeden karar endpoint’i sunma yaklaşımı etkili görünüyor. Bu ilgi benimseme kanıtı değil; verilen sonuçlar README’deki sınırlı benchmark’a dayanıyor.

ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenC

Spark-X2.5, yerel çalıştırma ve agent iş akışları için tasarlanmış 1.7B ve 4B boyutlarında açık dil modelleri serisidir.

Neden yükseliyor · Yeni model serisi; uzun context, küçük model boyutu ve agent/coding yeteneklerini aynı anda öne çıkarıyor. README'de 34 günlük yaş, 386 yıldız ve son 28 günde 9 commit ile 2 farklı yazar bilgisi var; bunlar erken ilgi ve güncel çalışma gösteriyor, ancak model performansını bağımsız olarak kanıtlamıyor.

Halo is an open-source framework built by White Circle for training large language and multimodal models

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how

Black-box LLM distillation for any verifiable task, with text or image inputs: teacher API -> rejection sampling -> SFT -> eval -> GRPO

ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuous

A MoE inference engine you can read in an afternoon.

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-awa

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Cal

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memo

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/a

Python package for LLM compression

Your AI intranet: network the computers you already own for inference and training.

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decodi

An open-source inference engine for AI functions in SQL

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM i

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

📚 An educational, step-by-step implementation of LLM KV cache reuse — learn prefix caching, chunk keys, storage & eviction, from pure Python

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Hemmingway-1: a 27B model that writes the way a person writes. Open weights, free for non-commercial use (CC BY-NC 4.0); commercial use by a

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

Open, calibrated, single-pass typed-decision models (4B & 27B) for finance and trading

Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVID

A small chat model trained from scratch: tokenizer → pretraining → SFT → RL → multimodal. 0.78B params, CORE 0.248, tool calling, vLLM. Weig

The open-core AI workbench — notebooks, agents, RAG, voice, and images across any model: OpenAI, Anthropic, Google, xAI, or local via Ollama

Self-hosted vLLM made simple. Git-friendly YAML for models and profiles, full lifecycle from an intuitive CLI and TUI.

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Disaggregated quantization official codebase

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

one-year LLM serving trace from Chutes

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hos

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Turn documents into structured JSON with local-first document AI. Run 100% locally by default, with API, CLI, and Web UI.

Network-faithful simulation of LLM serving and training deployments

Self-hosted, distributed, multi-model LLM inference

AI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NP

Open-source, self-hostable social listening.

Think only when the tool call looks unsure: confidence-gated reasoning for small local models. Code, BFCL results and paper.

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind

Plug-and-play, dependency-free web command center for DGX Spark fleets (and any GPU boxes): live per-node GPU/CPU/mem/temps/power, optional

Instinct: single-forward-pass decision models (choice / true-false / score) returning a probability per candidate

RigMark benchmarks local AI the way coding agents actually use it.

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

Composable LLM decoding on the probability simplex: library and paper code.

Declarative Python SDK for building AI inference platforms on Kubernetes — bare machines to a served model, composed rather than hand-assemb