Havuz

SkillCorpus, dağınık SKILL.md dosyalarını agent'lar için seçilmiş, aranabilir skill corpus'larına dönüştüren ve retrieval ile evaluation araçları sunan açık kaynak bir altyapıdır.

Neden yükseliyor · Agent'lar için paylaşılan ve göreve göre seçilen skill katmanına ilgi var; repo kısa sürede 667 yıldıza ulaşmış ve son 28 günde 9 commit ile 3 farklı yazar katkı vermiş. README ayrıca birden fazla benchmark'ta iyileşme bildiriyor; bu sonuçları bağımsız olarak doğrulayamıyorum.

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Code-only runtime toolkit for cost-aware multi-agent organization and control

Swap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augment

Synthetic fraud graph generator for benchmarking graph-based fraud detection models in financial services.

Measurement harness for proxy providers, browser engines and scraping targets. Every claim carries the run it came from.

Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

A composable framework for reproducible evaluation of long-running agents.

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Compare two local Persian (Farsi) speech-to-text models side by side: a Persian Whisper fine-tune vs. general large-v3, on Apple Silicon.

AnyFlow - flow-matching video generation: training, inference and VBench evaluation for t2v and i2v models.

Move whole tables between databases fast — Postgres, MySQL, ClickHouse, BigQuery. Rust engine, one-line Python API, bounded memory.

Audio-Oscar is a multi-agent framework for generating long-form, controllable audio from complex audio scene descriptions.

Give Vercel AI SDK agents a real persistent filesystem with Amazon S3 Files.

Several coding agents, one repo, at the same time: a reproducible lab to measure how they coordinate, and Médula, a kernel that decides who

PromptLedger, prompt şablonlarını fingerprint ile sürümleyip aday yanıtları karşılaştıran ve sonuçları tekrarlanabilir raporlara dönüştüren bir Python aracı ile Go CLI sunuyor.

Neden yükseliyor · 25 günlük bir repo için 35 yıldız dikkat çekiyor; ancak son 28 günde yalnızca 1 commit ve 1 farklı yazar var. Bu nedenle ilginin kalıcı benimsemeye dönüştüğünü gösteren yeterli aktivite kanıtı henüz yok.

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Catches multilingual voice agents placing the wrong order. τ-bench style benchmark + CI gate for voice ordering agents in English and Hingli

Measures how much code AI coding agents add, from patches benchmarks already publish

Generate degraded speech datasets for noise-robust ASR benchmarking

面向 LLM 交易 Agent 的抗污染回测、确定性回放与训练轨迹生成，支持 A 股研究｜Leakage-resistant backtesting, deterministic replay & training trajectory generation for LLM tr

AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether

Advanced DNS benchmark (UDP, DoH, DoT) with scoring, latency tests and detailed reports

Cache-aware frequency sort: a header-only C++20 sort for low-cardinality integer arrays on x86-64.

Bittensor Subnet Repository maintainer agent + a GitHub-history benchmark that scores it — can an agent make the maintainer decisions a stro

Diagnosing policy grounding in LLM judges via causal erasure. Companion code to the AIES 2026 submission READ.

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-bound

Benchmark and compare Ollama models across local and cloud endpoints with rich, sortable tables.

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

Open-source benchmark for evaluating LLMs on domain-specific geological reasoning, AI for geology, and source-grounded geological knowledge.

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

Profine automatically profiles and optimizes PyTorch training jobs on real GPUs, delivering measurable speedups and lower GPU costs before t

Network benchmarking suite for DNS, HTTP, and SSL — with DNSSEC validation, DoH/DoT support, high‑concurrency testing, and enterprise‑grade