Design statistically rigorous ad tests
ads-testskillsetup L1★4,890
AgriciDaniel/claude-ads ↗What it does
Design and track A/B tests for ad creative
Best for
Ad teams avoiding "winner calling" errors and ensuring statistical rigor in creative testing.
Inputs
- · Control creative
- · Test hypothesis
- · Sample size target
Outputs
- · Test plan with variant specs
- · Statistical power calculation
- · Winner declaration criteria (p<0.05)
- · Test result tracker
Preconditions
- · Traffic volume sufficient for significance
- · Platform supports variant tracking
Failure modes
- · Sample size too small
- · Novelty bias in early windows
- · Underpowered test (β>0.20)
Trust signals
- · Sample size calculator (sequential testing)
- · Statistical power threshold (β=0.20 default)
- · P-value threshold (p<0.05)
- · Winner declaration rules (20%+ CI non-overlap)
- · Novelty bias window (first 3 days flagged)