Havuz

Reef, agent isteklerini sunma, feedback toplama, öğrenme döngüsü yürütme ve kabul edilen güncellemeleri sürümleyerek dağıtma işlevlerini bir araya getiren Python altyapısıdır.

Neden yükseliyor · Sürekli öğrenen agent'lar gündemde ve Reef, inference'tan feedback'e, eğitimden sürüm dağıtımına uzanan altyapıyı tek projede toplamayı hedefliyor. Verilen ölçümlerde 28 günde 6.122 yıldız, 362 commit ve 19 farklı yazar güçlü ilgi ve geliştirme faaliyeti gösteriyor; ancak bu veriler tek başına üretim kullanımını veya teknik başarıyı kanıtlamıyor.

Agent Mesh

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache co

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimiz

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interf

Inference-native Tokenmaxxing Agent Harness for Loop Engineering

Entrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark,

Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine

From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-platform, and built for the best possible us

Your AI intranet: network the computers you already own for inference and training.

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

NeMo-Speech.cpp is a lightweight C++ inference runtime for Speech models

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

Ultrafast Qwen3-TTS: sub-50 ms time-to-first-audio at 10 requests per second.

Frontier: A Discrete-Event Simulator for Modern LLM Serving

Provider-pluggable orchestration runtime for multi-model AI inference. ( Sakana Fugu style )

Tree-based speculative decoding for Apple Silicon (MLX). ~10-15% faster than DFlash on code, ~1.5x over autoregressive. First MLX port with

An AI-native distributed data plane built in Rust that supports high performance RPC, KV Cache, Message Queue, and File & Object Acceleratio

AI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accu

Evidence-aware on-prem LLM inference sizing and TCO calculator

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLL

Zero-leak online detection of LLM decoding corruption and Free Web Search for your AI agents! Catch repetition loops, language drift and gar

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment

FastCrest Tether: the OSS edge-to-cloud AI deploy CLI. Optimize, verify, deploy across Jetson, RTX, Apple Silicon, AMD. Hybrid edge-cloud in

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + sy

Distributed LLM inference fabric for heterogeneous local clusters. Features zero-copy SPSC ring buffers, automated tuning, and an OpenAI-com

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

CPU-only, pure-Rust implementation of LiquidAI's LFM2.5-8B-A1B LLM

Fast Gemma 4 inference in pure Java

Code samples for building AI applications on AWS with Amazon Bedrock, including Converse API, tool use, RAG with Knowledge Bases, guardrails

Production-ready, bleeding-edge, upstream-first vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a), verified against real models on

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kern

llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 3080.

CPU-optimized LLM inference engine (C)

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely release self-hosted models behind

Network-faithful simulation of LLM serving and training deployments

a LLM inference engine to run on consumer hardware

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

Run AI models on Apple Silicon and Raspberry Pi

Community benchmark database for running LLMs on Apple Silicon Macs

SGLang Docker image for AMD Strix Halo (gfx1151 / Radeon 8060S)

Local LLM inference engine written from scratch in Rust — hand-written AVX-512 assembly kernels, Metal & Vulkan compute shaders. Supports Qw

ML model ports (ASR · TTS · LLM · VLM · vision · audio codecs) on the rlx multi-backend compiler + runtime — CPU/Metal/MLX/CUDA/wgpu

Recomputation-free, context-independent KV caching for large language models

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embe

qMLX: Custom inference engine for Qwen 3.5 122B on Apple Silicon, extending MLX with hybrid attention support, SSD-backed KV cache, and RYS