Bursts of Color - The Power of Experiments

Last week I received a fun surprise in the mail: a signed copy of The Power of Experiments, a new book from my friend Mike Luca of HBS. The book is a nice roundup of experimentation in business and government over the past couple decades.

When we first started selling ads at Yelp, we had no idea what customers would be willing to pay, so we picked a number and priced our initial packages at $50 per month. We wondered if the market would bear more, so the next month we raised prices to $100. To our sales team’s surprise, new customer signups actually increased at this higher price. So the next month we tried $150, then $200, then $300… where we found a good equilibrium point that held for a couple years. Of course this testing wasn’t very scientific, as we had few sales people and no real control group. Nonetheless, the results were pretty obvious (ACV up 6x with no decrease in velocity!), so it taught us the value of testing even when it’s the “quick and dirty” kind.

The beauty of experiments in human systems is that people so often surprise us. In fact, they often do the opposite of what rational economics says they should… like the pricing example above. I find behavioral economics endlessly fascinating for this reason. A few of my favorite books on this topic are:

A few years ago Mike Luca (author mentioned above) and I co-wrote this article about Thinking Like a Behavioral Economist. The headline is that all founders and executives are choice architects for their teams and users, whether they want to be or not: