Havuz

Can AI agents rebuild websites through browser exploration?

DR Pixel: Page-Native Deep Research with Vision-Language Models — benchmark, evaluation harness and training code

IdeaTaste - A benchmark for research taste: let LLM agents propose research ideas, then check how many of them the research area actually we

:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a

dsh-model-master（模型选择大师）— model selector for DeepSeek Harness: search across providers, probe any model with a real streaming request to mea

Measures which GGUF quant of a model to run on your GPU: real speed and quality loss (KL divergence), not estimates.

Reproducible speech-to-text benchmark: WER, latency and cost for Velma, Deepgram, AssemblyAI, Chirp 3, MAI-Transcribe 2, Moonshine and whisp

Swap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augment

The Virtual Filesystem Interface for AI Agents

A benchmark of AI QA agents finding real bugs in 4 open-source web apps, with 1,396 labeled bugs.

A benchmark for AI agents as teammates. Paper: https://arxiv.org/abs/2611.02633

DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with

A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The ga

Benchmark of AI coding agents building web apps and deploying them to Cloudish, DigitalOcean, Render and Vercel, measuring success, time, to

Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

Measures how much code LLM agents waste. 5 tasks, sandboxed, weekly snapshots, contestable references.

Official code for InfoVis: one 9B model for slide image generation and editing, with the VisSlideBench benchmark and 15M+ training records.

Can an AI agent fix production without breaking it? 37 live-server tasks, each with a hidden trap, graded on success and on harm to users.

Benchmark aberto de LLMs para o processo legislativo brasileiro

🔥 Orbia: Do Generated Videos Form a Consistent 3D World?

PaperBench High-Difficulty and Expanded Tasks: paper reproduction beyond machine learning. 93 tasks in 12 research areas, run in domain-nati

A platform for running and comparing harness self-improvement methods.

How much does an image model change what you told it to leave alone? Chained-edit benchmark for image models + installable scorer and Claude

The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it

Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

AnyFlow - flow-matching video generation: training, inference and VBench evaluation for t2v and i2v models.

AI API 中转站评测排行与切换工具：实测可用率、延迟与价格 · Open-source AI API relay benchmark, rankings & switcher for Claude Code / Codex

SWE-sweep, agent'ların büyük kod tabanlarındaki konumu ve türü önceden belirtilmeyen birden fazla bug'ı bulup düzeltebilmesini ölçen bir benchmark'tır.

Neden yükseliyor · 1 günlük, 33 yıldızlı yeni bir Facebook Research benchmark'ı olması ve LLM agent değerlendirmesine yönelik güncel bir soruyu ele alması dikkat çekebilir. Ancak yükselişin kesin nedeni bilinmiyor; son 28 günlük commit ve farklı yazar verisi ölçülmemiş.

SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation

Agent Smith, Codex görevleri için yerel model yönlendirmesi, macOS kullanım izleme arayüzü ve tekrarlanabilir benchmark araçları sunuyor.

Neden yükseliyor · Altı günde 194 yıldız alması dikkat çekici; yerel model routing, görünür kullanım takibi ve benchmark sonuçlarını tek projede birleştirmesi ilgi çekmiş olabilir. Ancak bu erken ilgi benimseme kanıtı değil: son 28 günde 5 commit ve 1 farklı yazar var.

Several coding agents, one repo, at the same time: a reproducible lab to measure how they coordinate, and Médula, a kernel that decides who

Catch AI agents that say "done" when the task isn't. Checks agent-trace/v1 traces with claim-evidence rules, LLM judges and a trained classi

一行安装的中文求职 Agent Skills 套件（岗位搜索/简历匹配/面试陪练/查公司）+《中文 AI 求职 Skill 公开横评》数据。Claude Code·Codex·MCP

Benchmark local LLM serving by replaying real agent trajectories

Placebo-controlled benchmark of the most-starred coding agent skills

🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Repro

同一份提示词，两座汴京：Claude Opus 5.5 与 GPT 6 Sol 各自从零写出的《清明上河图》实时 3D 网页 · Same prompt, two cities — a real-time 3D Bianjing benchmark for LLMs

DolphinBench - Mapping the Pareto frontier of Agent Memory

Real-time AI model power rankings (水墨战力榜) — usage, benchmarks, price-performance & speed from OpenRouter, as an ink-wash Streamlit dashboard

[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluatio

Laya (open source, local) vs TypeSafe Jev (API): two AI models race in Snake and fight in a Mortal-Kombat-style arena. Every move is a real

A Minesweeper benchmark for LLM agents. One identical board, up to nine models in parallel, one clock, one tool layer.

Measures how much code AI coding agents add, from patches benchmarks already publish

Glyd, model ağırlıklarını ve KV cache'i bit düzeyinde kayıpsız biçimde sıkıştırıp GPU'da kullanmayı hedefleyen bir compression yazılımıdır.

Neden yükseliyor · README, yaygın bir maliyet sorununa doğrudan yanıt verdiğini ve 32B ile 72B modellerde daha az GPU ile çalışmaya dair ölçümler sunuyor; ancak 9 günlük repoda 170 yıldız ve 474 commit'in yalnızca 2 farklı yazardan gelmesi, ilginin ve geliştirme temposunun geniş bir katkıcı tabanına dayandığını göstermiyor. Yükselişin kesin kaynağı veya bağımsız benimsenme verisi bilinmiyor.

Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAG

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verifi

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙

DrugTargetWorld: a synthetic biobank for training and benchmarking AI scientists on end-to-end drug target discovery. Paper: arXiv:2610.0955

AI Model World, yüzlerce AI modelini fiyat, yetenek, benchmark ve yayın tarihi verileriyle karşılaştıran piksel-art görselli bir web sitesidir.

Neden yükseliyor · Güncel model karşılaştırması ve görsel sunum, hızla değişen AI ekosisteminde ilgi çekici bir kullanım sunuyor. 11 günde 191 yıldız ve son 28 günde 56 commit dikkat çekici bir başlangıç; ancak iki farklı yazar bulunması ve kısa geçmiş, benimsemenin kapsamı ya da sürdürülebilirliği hakkında henüz güçlü kanıt oluşturmuyor.

Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.