Havuz

Track 42,331+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.

A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.

[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluatio

Runtime evidence that helps agents trace, profile, and burn down hotspots in application and native code, GPU kernels, and inference stacks.

SWE-sweep, agent'ların büyük kod tabanlarındaki konumu ve türü önceden belirtilmeyen birden fazla bug'ı bulup düzeltebilmesini ölçen bir benchmark'tır.

Neden yükseliyor · 1 günlük, 33 yıldızlı yeni bir Facebook Research benchmark'ı olması ve LLM agent değerlendirmesine yönelik güncel bir soruyu ele alması dikkat çekebilir. Ancak yükselişin kesin nedeni bilinmiyor; son 28 günlük commit ve farklı yazar verisi ölçülmemiş.

A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant k

HEWN 2.0 2026: AI Output Router for Precision Summaries & Polished Code

Free, open-source Blender AI plugin for the tools you already use. Codex and Claude Code installers, local MCP tools, editable .blend files

Real-world percentile benchmark for LLM inference servers: 1% A/B resolution, offline HTML reports. I am now on X also: https://x.com/bkuype

CLI Noise-aware CPU pinning

Unified Python/CLI time-series decomposition library with native acceleration, multivariate workflows, and machine-readable outputs.

A lightweight, general-purpose harness for tool-using LLM agents, fair benchmark evaluation, harness baselines, and personal assistant workf

Specification-driven multi-agent systems with OpenTelemetry observability, agent traces, benchmarking, MCP, and A2A.

Dashboard for InferenceX™, Open Source Continuous Inference | InferenceX 仪表板

Iterative agent harness improvement: run a coding agent on a hard task, generate the reusable tooling it was missing, qualify it, and replay

Is your project as good as the best out there? sota-scan benchmarks your repo against real top projects and hands you a scorecard + to-do li

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable

Pre-inference tool and agent routing for MCP, A2A and tool-rich LLM systems. Reduce candidate tools before model inference with policy-aware

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducib

Coding Agent Runtime & Evaluation Harness for controlled repository-level software repair

Cross-platform .NET performance engineering skill for coding agents, covering CPU, memory, GC, benchmarking, concurrency, startup, native pr

Compute-for-memory research lab for running larger LLMs on memory-constrained consumer GPUs. Official website exvram.pro

A pure-Python structured memory benchmark for multi-agent LLM systems — context graph vs vector RAG vs raw history dump, five scenarios, 18

LLM benchmark results for THOR forensic finding triage quality

A recursive self-improvement (RSI) workflow that evolves both target agents (e.g., Pi and DeepSeek Harness) and the RSI system itself.

Semantic cache strategy bench: replay recorded request traces through candidate caching policies and measure which one actually pays off.

Run a command and report its process tree's CPU, memory, and I/O usage

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

Auditable context capsules for LLM handoffs, coding agents, and OpenCode MCP workflows.

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decodi

Automated auditing pipeline for LLM and agent benchmarks — surfaces task ambiguity, environment conflicts, and evaluation bugs.

Occam's razor for Claude Code and Codex: a 30-line rule set. Benchmarks vs base: 51-55% lower estimated task cost on Opus/Sonnet 5.5 max wit

Open benchmark for expert collapse and routing efficiency in sparse MoE LLMs

Browser-native, GPU-accelerated editor for very large hierarchical 2D IC-layout scenes, written in Rust (native and WebAssembly).

BANTAM FACTORY — Your model. Your hardware. A coding factory that can improve itself.

Production-grade LLM Evaluation & Benchmarking Framework - GPT-4, Claude, Gemini, Mistral. Accuracy, latency, cost, hallucination, reasonin

Local workload scheduler for autonomous agents sharing an Apple Silicon Mac.

The open testing standard for voice AI agents. Deterministic + semantic + RAG augmented evaluation. Local first. Zero telemetry.

Claude Code plugin that makes coding agents measurably cheaper over time: collect token costs, distill candidate rules, benchmark them on a

Open browser arena for embodied / VLA stacking benchmarks

Benchmarks for pdf redaction tools