Havuz

Reef, agent isteklerini sunma, feedback toplama, öğrenme döngüsü yürütme ve kabul edilen güncellemeleri sürümleyerek dağıtma işlevlerini bir araya getiren Python altyapısıdır.

Neden yükseliyor · Sürekli öğrenen agent'lar gündemde ve Reef, inference'tan feedback'e, eğitimden sürüm dağıtımına uzanan altyapıyı tek projede toplamayı hedefliyor. Verilen ölçümlerde 28 günde 6.122 yıldız, 362 commit ve 19 farklı yazar güçlü ilgi ve geliştirme faaliyeti gösteriyor; ancak bu veriler tek başına üretim kullanımını veya teknik başarıyı kanıtlamıyor.

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimiz

Agent Mesh

Inference-native Tokenmaxxing Agent Harness for Loop Engineering

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache co

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark,

Ultrafast Qwen3-TTS: sub-50 ms time-to-first-audio at 10 requests per second.

NeMo-Speech.cpp is a lightweight C++ inference runtime for Speech models

Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine

Your AI intranet: network the computers you already own for inference and training.

From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-platform, and built for the best possible us

Provider-pluggable orchestration runtime for multi-model AI inference. ( Sakana Fugu style )

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

Evidence-aware on-prem LLM inference sizing and TCO calculator

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

Frontier: A Discrete-Event Simulator for Modern LLM Serving

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accu

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

An AI-native distributed data plane built in Rust that supports high performance RPC, KV Cache, Message Queue, and File & Object Acceleratio

Tree-based speculative decoding for Apple Silicon (MLX). ~10-15% faster than DFlash on code, ~1.5x over autoregressive. First MLX port with

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind

Distributed LLM inference fabric for heterogeneous local clusters. Features zero-copy SPSC ring buffers, automated tuning, and an OpenAI-com

AI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric

Network-faithful simulation of LLM serving and training deployments

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + sy

CPU-only, pure-Rust implementation of LiquidAI's LFM2.5-8B-A1B LLM

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kern

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

FastCrest Tether: the OSS edge-to-cloud AI deploy CLI. Optimize, verify, deploy across Jetson, RTX, Apple Silicon, AMD. Hybrid edge-cloud in

Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment

CPU-optimized LLM inference engine (C)

Production-ready, bleeding-edge, upstream-first vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a), verified against real models on

Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, re

A free textbook on building an LLM inference stack from scratch in Rust.

Fast Gemma 4 inference in pure Java

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

qMLX: Custom inference engine for Qwen 3.5 122B on Apple Silicon, extending MLX with hybrid attention support, SSD-backed KV cache, and RYS

Code samples for building AI applications on AWS with Amazon Bedrock, including Converse API, tool use, RAG with Knowledge Bases, guardrails

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen

llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.

Run AI models on Apple Silicon and Raspberry Pi

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding

Memory-aware GGUF inference runtime with OpenAI-compatible serving

SGLang Docker image for AMD Strix Halo (gfx1151 / Radeon 8060S)

Distributed, peer-to-peer, decentralized network for LLM inference — peers seed & leech completions, metered by BitTorrent-protocol-style up

Fast WebGPU runtime for 1-bit (binary-weight) LLMs in the browser. Bit-exact, zero runtime dependencies.