Havuz

Reproducible benchmark for sanctions name-screening effectiveness. Measures recall across 18 documented classes of name variation — translit

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

A composable framework for reproducible evaluation of long-running agents.

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-bound

SelfStarter, bir coding agent’a görev brief’i verip işi insan yönlendirmesi olmadan birden çok oturum boyunca yürütmesi için tasarlanmış erken aşama bir harness’tir.

Neden yükseliyor · 58 günlük bir repoda 387 yıldız, uzun süre kendi işini yürüten agent’lara yönelik ilgiyle uyumlu. README’deki Agent City örneği de fikri somutlaştırıyor. Ancak son 28 gündeki 1 commit ve 1 farklı yazar, güncel geliştirme temposu veya geniş katkı tabanı için güçlü kanıt sunmuyor.

MerchantBench is a 365-day, order-level benchmark for evaluating the long-term coherence of LLM agents in seller-side e-commerce operations.

Open-source benchmark for evaluating LLMs on domain-specific geological reasoning, AI for geology, and source-grounded geological knowledge.

An esoteric programming language built to break LLMs and coding agents.

Simplified Technical English for Code - extract, adapt, and enforce ASD-STE100 rules for software documentation. 9-agent pipeline, 59-test b

Compressed, action-first output style for Claude Code — a benchmarked merge of caveman and i-have-adhd, scored on quality as well as tokens.

Track 42,331+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.

StealthBench is a benchmark for evaluating whether autonomous agents operate with the operational security (OPSEC) discipline that separates

[NeurIPS 2026 Verify-Agents] SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

多智能体小说写作框架 | Multi-agent novel-writing framework — 每个角色都有自己的脑子,每一章都有质量门禁

Evidence-first infrastructure for reproducible, isolated executable evaluation and online rewards.

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

An auditable and reproducible evaluation suite for LeWM-compatible latent world models, with official-compatible and difficulty-controlled p

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

A 125-task benchmark for knowledge-mediated implicit associations and retrieval blind spots in long-term agent memory.

Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

A counterfactual battery for instruction following in vision-language-action policies: change one word of the instruction, hold the scene, a

How many simultaneous 4K transcodes can your GPU sustain? Multi-vendor (Intel/AMD/NVIDIA/CPU) benchmark for Unraid with HDR tone-mapping, su

aws-bench measures how well AI agents and model combinations perform on real AWS work — diagnosing misconfigurations, provisioning infrastru

HoloGeo: Evidence-driven VLM for robust image geo-localization and landmark-bias mitigation.

Scenarios and tasks for evaluating AI agents on real-world AWS automation tasks. Use the aws-bench framework (https://github.com/aws-bench/a

Continuous benchmarks for multi-device CoW filesystems (btrfs, ZFS, bcachefs): snapshots, aging, compression, redundancy layouts

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

[ACM MM 2026] Official implementation of Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

NEXBENCH measures what an autonomous agent can actually do on-chain — execute transactions, route swaps, bridge funds, manage DeFi positions

Move whole tables between databases fast — Postgres, MySQL, ClickHouse, BigQuery. Rust engine, one-line Python API, bounded memory.

Ứng dụng miễn phí kiểm tra sức khoẻ ổ đĩa & hệ thống cho macOS — SMART, nhiệt độ, benchmark tốc độ, phát hiện USB/SSD giả dung lượng. Cho cộ

PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.

Unified World Model Inference & Evaluation Infrastructure

Beyond Time Shifts (ECCV 2026): a reference-free audio-visual synchronization evaluator built on Qwen2.5-Omni-3B, with SynthSync data, R-GRP

Red Team AI Benchmark: Evaluating LLMs for authorized offensive-security tasks

Compare web access APIs (web unblockers / scrape APIs) head-to-head against a fixed suite of ~90 real-world, hard-to-scrape URLs

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

Official repository for RoboDojo

Cognitive-structured Multimodal Agent (CMA-Harness): a memory-centric agent for long-horizon multimodal understanding, generation, and editi

[ECCV 2026] AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World

Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audio ·

[ECCV 2026] Towards Spatial Supersensing in the Wild

Evidence profiles for AI-generated RTL that expose what functional benchmarks miss. Seeking collaborators, design partners, and validation

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity re

Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic sco

TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation

Advanced DNS benchmark (UDP, DoH, DoT) with scoring, latency tests and detailed reports

Bittensor Subnet Repository maintainer agent + a GitHub-history benchmark that scores it — can an agent make the maintainer decisions a stro