One question you should never ask

A while ago, we talked about how you should ask indirect questions rather than questions that are the literal inverse of the answer you’re trying to get.

I try not to harp on “bad” questions (like “would you pay for this” “is this a good idea”) and whatnot because it’s cognitively more effective to hear repetition of the things you should do rather than what you shouldn’t.

If you tell someone not to think about purple elephants, they will think about purple elephants, no matter how much you tell them not to think about purple elephants. This is also why telling someone who is nervous not to be nervous isn’t effective, either.

So I haven’t spent much time in this newsletter telling you what not to ask, and have instead talked about what you might ask instead.

But today we’re going to talk about a question you shouldn’t ask because it’s SO common. I bet you’ve seen it many times, and after reading this, it will jump out at you even more.

The flavor of question is: “What could we do differently?”

A few weeks ago, Mathieu Dhondt alerted to me an interesting paper on customer research: “The Cost of New Ideas: Idea Generators Become Less Satisfied.” The paper claimed that asking people for ideas on things a business could do better reduced their satisfaction with the business.

I had to get a copy. I don’t have journal access, so I emailed the author, Joshua H. Katz, and he kindly sent me a copy. (Most academics will do this.)

The paper tested idea generation among three different groups: consumers (local restaurant selection), employees, and users (in a very meta turn, rating Amazon Mechanical Turk, where the study took place).

The idea of the study was to see the effect of coming up with new ideas and whether it made someone like that original thing less than if they had just been asked how much they liked it.

In the initial control groups, people were asked to, say, list 10 different local restaurants, and then respond to questions like “Typically, I am happy with my experience at restaurants” on a on a five-point scale from “very dissatisfied” to “very satisfied.”

The experimental groups, meanwhile, were asked to come up with 10 new ideas. In the restaurant case, they needed to come up with 10 ideas for new restaurants in their city, and then answer the same satisfaction questions.

The results held up in subsequent experiments with negative questions (“Restaurants these days leave a lot to be desired”) and by mixing categories (people were asked to generate restaurant ideas and then rate their satisfaction with Mechanical Turk, and their satisfaction with Mechanical Turk did not decrease like it did for people who had to generate ideas for Mechanical Turk).

Whether it’s because coming up with new ideas makes the current situation seem sub-optimal or because you feel demoralized that your ideas will never become reality, this study makes a compelling case not to ask people for suggestions.

This whole newsletter is about understanding customer needs and listening to them to pull out their needs.

So it makes sense that if at this point, you’re saying to yourself, “wait, hold on, I shouldn’t ask customers for ideas?!”

My interpretation of this study is that you should still talk to customers—just avoid questions like:

(I myself stopped asking the “What can we do better?” question in surveys months ago for the simple reason that most people weren’t responding and when they did, their suggestions were rarely feasible or viable for us.)

I caveat this by saying it’s probably still okay to leave the door open to feature suggestions. For example, when recruiting existing customers for interviews, I’ve found it’s helpful to include something to the effect of “I’d also love to hear any feature suggestions you might have,” which does not assume or require them to have suggestions yet lets them know you’re open to it.

Instead of asking for ideas, ask for process.

Learn where they spend a lot of time, experience frustration, and spend money. Pulling that together with what is feasible and viable for you to create, you can then uncover ideas by talking to customers.

As the study showed, asking people to describe current options didn’t decrease satisfaction. So ask people to describe their current reality to you, and listen.

The book tour continues! I was on Justin Jackson’s Build Your SaaS podcast this week.

People seem to be enjoying the conversation. I hope you’ll take a listen!