Design and score marketing experiments
growth-engineskillsetup L1★2,362
ericosiu/ai-marketing-skills ↗Causal-lift measurements
ab-experimentation-27pp vs no-skill baselinewith-skill 73% · baseline 100%
Measured by running the task with and without this artifact, K=5, graded by deterministic checks — no LLM judging.
What it does
Design and score marketing experiments with statistical analysis
Best for
Teams running 5+ simultaneous A/B tests across channels who need automated winner detection with statistical rigor
Inputs
- · experiment hypothesis
- · variant definitions (2-10)
- · metric to track
- · target cycle hours
Outputs
- · experiment status (running/trending/keep/discard)
- · statistical significance (p-value, CI)
- · promoted best practices to playbook
Preconditions
Python 3, experiment data directory, telemetry consent
Failure modes
small sample sizes (<15 samples/variant) reduce statistical power; manual min-samples override needed for low-volume channels
Trust signals
- · Bootstrap confidence intervals + Mann-Whitney U test (formal stats)
- · batch mode supports 3-10 variants
- · auto-promotes winners (p<0.05 AND 15% lift) to living playbook
- · pacing alerts for campaign health monitoring