In my experience, I find that teams and organizations report many winning A/B tests with high uplifts, but somehow they don’t seem to bring those uplifts in reality. How come?
Five types of A/B test “wins” can exaggerate discovered uplifts. (Use the acronym “de FACTO”):
- False winners
- Changing winners
- Tricked winners
- Overestimated winners.
Do you experience the same in your organization?
Furthermore in this newsletter:
- Free Feature Flags and controlled rollouts in Optimizely
- Amazon culture of innovation
- Handbook: Design. Think. Make. Break. Repeat.
- “Hamel Hell” - Talk from Stéphane Hamel @ Superweek (video).
Have a nice week!