View profile

CRO weekly #8 - Segmenting A/B test results increases false positive rate

Good morning, In this week newsletter the third statistical ghost: multiple comparisons. As soon as
CRO weekly #8 - Segmenting A/B test results increases false positive rate
By Tom van den Berg • Issue #8 • View online
Good morning,
In this week newsletter the third statistical ghost: multiple comparisons.
As soon as you look at more than one metric (this can also be a segment), you increase your chances of seeing a false positive. To put it simple, every comparison you make has a chance of showing a false positive. So more comparisons means more false positives.
For example: if you segment your test to device type. There are 3 different devices: desktop, tablet and mobile. As a result the chance of a false positive increases from 5% to 15%.
What else in this newsletter:
  • The decoy effect: how are you influenced to choose without really knowing it.
  • The state of marketing training in 2019. Research done by ConversionXL.
  • Summary of the Friends of Search 2019 event.
  • New tool to measure sitespeed: speedmonitor. io
  • Conversie Specialist training @ Online Dialogue
Enjoy your week.
Cheers!
Tom

Reads
Training
Tools
Events
Did you enjoy this issue?
Tom van den Berg

CRO weekly: reads, events, talks, tools about conversion rate optimization

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue