Do you check for SRM in your A/B tests? No? I think you should! In the first article is explained why.
One of the most useful indicators of a variety of data quality issues is a Sample Ratio Mismatch (SRM) – the situation when the observed sample ratio in the experiment is different from the expected. While a simple statistical check is used to detect an SRM, correctly identifying the root cause and preventing it from happening in the future is often extremely challenging and time consuming.
Ignoring the SRM without knowing the root cause may result in a bad product modification appearing to be good and getting shipped to users, or vice versa.
What else in this newsletter:
- [DUTCH] Eerste hulp bij Sample Ratio Mismatch in je experimenten
- Live notes from the Growth Marketing Summit 2019 in Frankfurt
- A/B Testing Tools: VWO Compared to Google Optimize and Convert
- [VIDEO] How to build a Killer Growth Hacking Team
Job opportunities @ Videoland & Digital Power
Have a nice day.