Havuz

A benchmark for AI-driven CAD generation and editing

Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

NarrowWide: A LiDAR-inertial dataset with repeated transitions between confined and open spaces (IEEE T-FR 2026)

Ultra-Fast Python Solutions: 110x Speed Optimization Guide 2026

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity re

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGU

The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it,

Self-healing DNS optimizer for servers in Iran. Benchmarks 60+ DNS servers, installs a local caching resolver, auto-detects Docker, and runs

🚀 Automated scraper & injector for 100+ free AI coding models into 9router combos. Live pre-tested, ranked by coding benchmarks & zero-conf

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

A composable framework for reproducible evaluation of long-running agents.

WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

[NeurIPS 2026] SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

NEXBENCH measures what an autonomous agent can actually do on-chain — execute transactions, route swaps, bridge funds, manage DeFi positions

Anti-slop skills for AI agents: slopbeth cleans the shipped writing, slopgent cleans the conversation.

A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

Aim Genetics Lab: CRISPR Gene Editing & Bioinformatics Tools 2026

📖 Data Engineering from Raw Corpora

DrugTargetWorld: a synthetic biobank for training and benchmarking AI scientists on end-to-end drug target discovery. Paper: arXiv:2610.0955

RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)

Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Repro

Benchmark of AI coding agents building web apps and deploying them to Cloudish, DigitalOcean, Render and Vercel, measuring success, time, to

A benchmark for evaluating LLM × harness performance.

Physical-Engineering’s Last Exam

Offline cyber security agent for your apps and AI coding tools. Open R&D: measured in public, improved every release.

Laya (open source, local) vs TypeSafe Jev (API): two AI models race in Snake and fight in a Mortal-Kombat-style arena. Every move is a real

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Specul

Agent Memory Benchmark

Evaluating frontier models on real-world ai-infra engineering tasks

SciPIvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Does your AI hydrologic model conserve mass, energy and momentum?

AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?

SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation

OptiScaler ILION — Community fork with bilingual ES/EN interface, FSR 4 version switching, live performance metrics and benchmarking tools.

🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure

Measures how much code LLM agents waste. 5 tasks, sandboxed, weekly snapshots, contestable references.

Honest local LLM deployment planning and benchmarking for high unified-memory Macs and future Linux/NVIDIA rigs.

Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured

NeurIPS2026 , Scalable annotation pipeline for action-aglined fine-grained instruciton for Visual-language-Action model

Reproduction package for "Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity" — ten agent arms across AppWorld an

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

Benchmark AI Agents on Enterprise Workflows

Evaluation code for SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images (NeurIPS 2026).

State-Aware Collaboration Protocol: auditable work receipts for AI agents.

Benchmarking repository-borne prompt injection attacks and lightweight defenses for local coding agents. DL4C @ ICML 2026.

Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward-mo

NEEDLE. A live search and retrieval benchmark from Keenable.ai