Havuz

Ten frontier VLM agents, five robotic capabilities, 1,000 cases. GeoProbe and traceable analyses of geometry, spatial reasoning, affordance,

PromptLedger, prompt şablonlarını fingerprint ile sürümleyip aday yanıtları karşılaştıran ve sonuçları tekrarlanabilir raporlara dönüştüren bir Python aracı ile Go CLI sunuyor.

Neden yükseliyor · 25 günlük bir repo için 35 yıldız dikkat çekiyor; ancak son 28 günde yalnızca 1 commit ve 1 farklı yazar var. Bu nedenle ilginin kalıcı benimsemeye dönüştüğünü gösteren yeterli aktivite kanıtı henüz yok.

A Open-Source Search Feature.

Reproducible benchmark for sanctions name-screening effectiveness. Measures recall across 18 documented classes of name variation — translit

Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the

Benchmarking data agents for verifiable analytics over heterogeneous workspaces.

Official benchmark reproduction code for the SAG paper

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

Benchmark for the quality of LLM-generated test suites — anti-fragility, rigor, mocking discipline, reuse — scored against human baselines,

Live Benchmark for long Horizon tool using Agentic Systems

Official code for "CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges"

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, sa

How many simultaneous 4K transcodes can your GPU sustain? Multi-vendor (Intel/AMD/NVIDIA/CPU) benchmark for Unraid with HDR tone-mapping, su

📊 Automated hourly benchmarks for 20+ NVIDIA NIM models — interactive dashboard, zero infra, self-hostable. Open-source & community-driven.

Catches multilingual voice agents placing the wrong order. τ-bench style benchmark + CI gate for voice ordering agents in English and Hingli

Measures how much code AI coding agents add, from patches benchmarks already publish

Open benchmark datasets republished in stressed form to evaluate ML/LLM robustness. Curated by Santander AI Lab.

Red team vs blue team benchmark for scientific paper verification. Plant defects into open access biomedical papers, build gates to catch th

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Open-source benchmark of 5 intentionally vulnerable web apps with 100 baseline vulnerabilities for evaluating AI penetration testing agents.

LLM benchmark and leaderboard for narrator-bias sycophancy, opposite-narrator contradictions, and judgment consistency.

Benchmarking Agents on Real-World Terminal Tasks

PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

CoDA-Bench is a benchmark for code agents on data-intensive tasks. 🎈代码智能体能搞定数据密集型任务吗?

AMBER (sealed-in-amber historical replay evaluation): real-event replay, physical sealing, pre-registered scoring — method spec; cases priva

PureDocBench: source-traceable benchmark for document parsing across clean, degraded, and real-world settings

Lynn GitHub 镜像仓 · Qwen3.8-27B Coder390 EfficientThink 本地 GGUF（llama.cpp Q2/Q3，内置 Q4 MTP）· 主仓 https://github.com/MerkyorLynn/Lynn · 下载 https:

Benchmarks for pdf redaction tools

Generate degraded speech datasets for noise-robust ASR benchmarking

Generative UI benchmark

AgentBehaviorBench (ABB): A benchmark for evaluating AI agents’ ability to test other agents and detect behavioral defects.

Agent Smith, Codex görevleri için yerel model yönlendirmesi, macOS kullanım izleme arayüzü ve tekrarlanabilir benchmark araçları sunuyor.

Neden yükseliyor · Altı günde 194 yıldız alması dikkat çekici; yerel model routing, görünür kullanım takibi ve benchmark sonuçlarını tek projede birleştirmesi ilgi çekmiş olabilir. Ancak bu erken ilgi benimseme kanıtı değil: son 28 günde 5 commit ve 1 farklı yazar var.

The Humanness Index: the open benchmark for how human voice AI sounds, built by Vapi. Blind same-voice battles, pure-vote Elo, with a real-h

AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results

🌟 [Survey & Benchmark] A unified taxonomy, geometric framework, and cross-domain benchmark for modern optimizers

Local-first CLI for benchmarking LLMs on real hardware — quality, speed, reliability, and a real multi-turn agent loop.

LintEval: Benchmarking LLMs on Linter-Guided File Refactoring (EMNLP 2026 Findings) — multi-language benchmark for linter-guided refactoring

A temporal visual reasoning benchmark over egocentric robot manipulation video — 39 scenarios, 7 dimensions, 34,713 questions

Compare web access APIs (web unblockers / scrape APIs) head-to-head against a fixed suite of ~90 real-world, hard-to-scrape URLs

What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here

面向 LLM 交易 Agent 的抗污染回测、确定性回放与训练轨迹生成，支持 A 股研究｜Leakage-resistant backtesting, deterministic replay & training trajectory generation for LLM tr

[ACL 2026] PIArena: A Platform for Prompt Injection Evaluation

Benchmark and correctness oracle for matching engines, with CI/CD checks against 100 billion order messages of precomputed consensus. Five

UI-UX: a 4B multimodal LLM for mobile UX defect diagnosis, with UXBench — the first vision–language benchmark for UX reasoning (CVPR 2026 Fi

Beyond Time Shifts (ECCV 2026): a reference-free audio-visual synchronization evaluator built on Qwen2.5-Omni-3B, with SynthSync data, R-GRP

[NeurIPS 2026 Meta-Agents] PostTrainBench⁰ — Can LLM agents automate LLM post-training without gradients?

HoloGeo: Evidence-driven VLM for robust image geo-localization and landmark-bias mitigation.

An arena where LLMs write robot controllers and their code competes.

Daily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜

MODA: open fashion retrieval benchmark and models by Hopit AI. MODA (203M, open source), MODA Pro Lite (213M, open weights), MODA Pro (hoste