Benchmark skill effectiveness with controlled variables

Benchmark AI agent skills with controlled-variable A/B testing

A/B testing AI agent skills or deck configurations with controlled variables, native judge scoring, and parallel isolated execution.

• · Deck TOML file (skill definitions)
• · Brief task description or prompt
• · Optional explicit player (kimi/codex/claude)

• · Benchmark report with comparative scores
• · Artifact outputs from each side
• · Judge score and reasoning

• · npm/bun (Node.js)
• · Firecrawl or web scraper (optional for fetch-based tests)

• · Deck TOML file with valid skill definitions
• · Arena.toml for vs mode with side definitions

• · Invalid TOML syntax causes parse failure
• · Cross-player mode requires different agent binaries installed
• · Timeout if task exceeds --timeout value

• · Agent-orchestrated by default (parallel subagents), cross-player mode via Bun.spawn
• · Isolated workdirs with deck link + skill preparation
• · Native judge scoring (not external service)