In the coming weeks I will share a couple of statistical ghosts in experimentation shared by Skyscanner. Experimentation is all about causality, but getting causality right is easy to get wrong! In this series of blog posts Skyscanner highlights some counter intuitive ways to arrive at the wrong conclusion with experiments.
This week starts with: It’s either significant or noise
After running an experiment we want to understand if any differences observed are random noise, or real. Unfortunately, confirmation bias demonstrates that we tend to digest data in ways that confirms what we already think.
Furthermore in this newsletter: why you need an executive sponsor for experimentation program success / Superweek summary / GTM - G+ Archive.
Enjoy your week.