Havuz

Evaluation code for SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images (NeurIPS 2026).

Several coding agents, one repo, at the same time: a reproducible lab to measure how they coordinate, and Médula, a kernel that decides who

What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.

MathPinpoint: problem-level math retrieval dataset. Find the web page that solves the same problem under the same conditions, correctly. 3.5

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

Ten frontier VLM agents, five robotic capabilities, 1,000 cases. GeoProbe and traceable analyses of geometry, spatial reasoning, affordance,

Benchmark local LLM serving by replaying real agent trajectories

Placebo-controlled benchmark of the most-starred coding agent skills

A Open-Source Search Feature.

HAEnv — synthetic longitudinal patients with code-derived gold, for evaluating health agents over time

Chinese-first LLM jailbreak benchmark & red-team toolkit | 中文LLM安全评测框架 | by POJIA.AI

RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)

MathmoBench, kombinatoryal matematik sorularında modelleri makineyle doğrulanabilir çözüm ve imkânsızlık kanıtları üzerinden değerlendiren bir benchmark ve doğrulama aracıdır.

Neden yükseliyor · AI evaluation odağı ve sertifika tabanlı, deterministik puanlama yaklaşımı dikkat çekebilir. Dört günlük repo için 85 yıldız erken ilgi gösteriyor; ancak 2 commit ve 1 farklı yazar, benimseme veya sürdürülebilirlik kanıtı değil.

🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Repro

An open benchmark for what consent banners cost the pages they sit on — time to banner, payload, viewport blocked and page stability, measur

同一份提示词，两座汴京：Claude Opus 5.5 与 GPT 6 Sol 各自从零写出的《清明上河图》实时 3D 网页 · Same prompt, two cities — a real-time 3D Bianjing benchmark for LLMs

JevGym is an open-source platform designed to benchmark and facilitate better probabilistic estimation in Jev-alike models

DolphinBench - Mapping the Pareto frontier of Agent Memory

Real-time AI model power rankings (水墨战力榜) — usage, benchmarks, price-performance & speed from OpenRouter, as an ink-wash Streamlit dashboard

[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluatio

[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.

[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset

Best Open Source Aim Trainer Benchmark Tool 2026

Aim Genetics Lab Open Research Tools 2026: Build Precision Breeding Models

Laya (open source, local) vs TypeSafe Jev (API): two AI models race in Snake and fight in a Mortal-Kombat-style arena. Every move is a real

A Minesweeper benchmark for LLM agents. One identical board, up to nine models in parallel, one clock, one tool layer.

Measures how much code AI coding agents add, from patches benchmarks already publish

Benchmarks for pdf redaction tools

Glyd, model ağırlıklarını ve KV cache'i bit düzeyinde kayıpsız biçimde sıkıştırıp GPU'da kullanmayı hedefleyen bir compression yazılımıdır.

Neden yükseliyor · README, yaygın bir maliyet sorununa doğrudan yanıt verdiğini ve 32B ile 72B modellerde daha az GPU ile çalışmaya dair ölçümler sunuyor; ancak 9 günlük repoda 170 yıldız ve 474 commit'in yalnızca 2 farklı yazardan gelmesi, ilginin ve geliştirme temposunun geniş bir katkıcı tabanına dayandığını göstermiyor. Yükselişin kesin kaynağı veya bağımsız benimsenme verisi bilinmiyor.

Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAG

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verifi

DrugTargetWorld: a synthetic biobank for training and benchmarking AI scientists on end-to-end drug target discovery. Paper: arXiv:2610.0955

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor.

AI Model World, yüzlerce AI modelini fiyat, yetenek, benchmark ve yayın tarihi verileriyle karşılaştıran piksel-art görselli bir web sitesidir.

Neden yükseliyor · Güncel model karşılaştırması ve görsel sunum, hızla değişen AI ekosisteminde ilgi çekici bir kullanım sunuyor. 11 günde 191 yıldız ve son 28 günde 56 commit dikkat çekici bir başlangıç; ancak iki farklı yazar bulunması ve kısa geçmiş, benimsemenin kapsamı ya da sürdürülebilirliği hakkında henüz güçlü kanıt oluşturmuyor.

LLM gateway comparison: self-hosted and managed AI API gateways compared on protocol compatibility, billing units, timeout and retry semanti

Live Benchmark for long Horizon tool using Agentic Systems

Red team vs blue team benchmark for scientific paper verification. Plant defects into open access biomedical papers, build gates to catch th

LynnReal-Omni, video üretimi, kontrolü, düzenlemesi ve onarımı için farklı görevleri tek bir multimodal diffusion modelinde birleştiren erken beta bir framework’tür.

Neden yükseliyor · 14 günlük yaşına karşılık 230 yıldız, son 28 günde 75 commit ve 5 farklı yazar kayda değer erken ilgi ve geliştirme faaliyetine işaret ediyor. Çok sayıda video görevini tek modelde birleştirme ve üç/dört adımlı generation iddiaları dikkat çekici; ancak README projenin erken beta olduğunu belirtiyor ve bu performans iddiaları burada bağımsız olarak doğrulanmış değil.

Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

Qwen3.8-27B on RTX 5090 Laptop 24GB — TWIN-TURBO NVFP4 (native llama.cpp) vs Coder390 Q3 LynnStyle (.ninfer + NInfer engine in WSL2/Docker):

A temporal visual reasoning benchmark over egocentric robot manipulation video — 39 scenarios, 7 dimensions, 34,713 questions

Did your change make your LLM server faster? Measure it: faster, slower, or too close to call. Linux and Apple Silicon.

Your coding agent paints pixel art in the Aseprite window you already have open. MCP server + skills for Claude Code, omp, Codex, Gemini CLI

[NeurIPS 2026 Meta-Agents] SAEScientist-Bench: Can AI agents conduct autonomous SAE interpretability research?

A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence

Reproduction package for "Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity" — ten agent arms across AppWorld an

Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured

RigMark benchmarks local AI the way coding agents actually use it.

An arena where LLMs write robot controllers and their code competes.