View profile

CRO weekly #36 - Why you should check for Sample Ratio Mismatch in A/B tests

Good morning, Do you check for SRM in your A/B tests? No? I think you should! In the first article is
CRO weekly #36 - Why you should check for Sample Ratio Mismatch in A/B tests
By Tom van den Berg • Issue #36 • View online
Good morning,
Do you check for SRM in your A/B tests? No? I think you should! In the first article is explained why.
One of the most useful indicators of a variety of data quality issues is a Sample Ratio Mismatch (SRM) – the situation when the observed sample ratio in the experiment is different from the expected. While a simple statistical check is used to detect an SRM, correctly identifying the root cause and preventing it from happening in the future is often extremely challenging and time consuming.
Ignoring the SRM without knowing the root cause may result in a bad product modification appearing to be good and getting shipped to users, or vice versa.
What else in this newsletter:
  • [DUTCH] Eerste hulp bij Sample Ratio Mismatch in je experimenten
  • Live notes from the Growth Marketing Summit 2019 in Frankfurt
  • A/B Testing Tools: VWO Compared to Google Optimize and Convert
  • [VIDEO] How to build a Killer Growth Hacking Team
Job opportunities @ Videoland & Digital Power
Have a nice day.
Cheers,
Tom

Reads
Job opportunities
Did you enjoy this issue?
Tom van den Berg

CRO weekly: reads, events, talks, tools about conversion rate optimization

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue