Havuz

Reef, agent isteklerini sunma, feedback toplama, öğrenme döngüsü yürütme ve kabul edilen güncellemeleri sürümleyerek dağıtma işlevlerini bir araya getiren Python altyapısıdır.

Neden yükseliyor · Sürekli öğrenen agent'lar gündemde ve Reef, inference'tan feedback'e, eğitimden sürüm dağıtımına uzanan altyapıyı tek projede toplamayı hedefliyor. Verilen ölçümlerde 28 günde 6.122 yıldız, 362 commit ve 19 farklı yazar güçlü ilgi ve geliştirme faaliyeti gösteriyor; ancak bu veriler tek başına üretim kullanımını veya teknik başarıyı kanıtlamıyor.

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine f

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Run Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime

ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenC

Local inference for diffusion language models (LLaDA, Dream), written in Mojo

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how

llama.cpp fork that abliterates at inference

pooled, birden fazla cihazın tarayıcı GPU’larını WebRTC üzerinden birleştirerek büyük open model’leri birlikte çalıştırmasını sağlayan bir LLM inference projesidir.

Neden yükseliyor · 27 günlük yaşına göre 461 yıldız ve son 28 günde 90 commit güçlü bir erken ilgi ve geliştirme temposuna işaret ediyor. Tarayıcıdan katılımla cihazları birleştirme fikri, yerel inference ve donanım paylaşımı ilgisini çekebilir; ancak yalnızca 2 farklı yazar olduğundan aktivite geniş bir katkıcı topluluğunun kanıtı değil. İlginin nedenini kesin olarak belirlemek mümkün değil.

Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX cards (tuned on 3080 10gb). A measured, tuned and benchmarked fork of Str

Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVID

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimiz

Agent Mesh

Pure-Rust MoE inference engine for Qwen3.5/3.6/3.8 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp

🧱 Experimental library for building inference engines

ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuous

A MoE inference engine you can read in an afternoon.

Inference-native Tokenmaxxing Agent Harness for Loop Engineering

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache co

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

Benchmark local LLM serving by replaying real agent trajectories

AI inference gateway and control plane for multi-provider routing, policies, caching, tools, and observability.

Feel how fast a local LLM runs before you buy the hardware

Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark,

Ultrafast Qwen3-TTS: sub-50 ms time-to-first-audio at 10 requests per second.

Bilingual English and Mandarin text to speech in C++ and GGUF. Voice design, cloning, direction and experimental voice conversion. Vulkan ba

NeMo-Speech.cpp is a lightweight C++ inference runtime for Speech models

Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine

Your AI intranet: network the computers you already own for inference and training.

An open-source inference engine for AI functions in SQL

Serve your models fast on your own infra in 3 steps.

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-platform, and built for the best possible us

Qwen3.8-Flash-Next (180B MoE) on one 12-16 GB NVIDIA GPU, Linux or Windows. Own engine, own compression.

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

Evidence-aware on-prem LLM inference sizing and TCO calculator

Runs Hugging Face ONNX models on any Nx backend, with no per-architecture code

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Ra

Provider-pluggable orchestration runtime for multi-model AI inference. ( Sakana Fugu style )

Frontier: A Discrete-Event Simulator for Modern LLM Serving

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, on

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

BOAR - Adaptive Local Intelligence

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accu

Realtime dashboard for the Halogen LLM inference server

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU