View profile

Debugging Interviews

Deploy Empathy
Deploy Empathy
This was written as part of the rough draft for Deploy Empathy, a practical guide to interviewing customers.
Like it? Order the book:
—————————————
Every interview is an adventure in its own right.
You might walk into it hoping to find one thing, and end up learning something completely different that you’ve never come across.
This is genuinely one of my favorite parts about it.
But sometimes they’re an adventure for reasons that aren’t so exciting.
You get one-word responses. A person is rude. They tell you they only have 10 minutes when you’d scheduled 30. They talk about something that’s entirely unrelated to what you hoped to hear about.
This happens, and I want you to know that’s normal for things to go a little sideways sometimes. That’s okay, and isn’t a reflection on your own skills.
We all know that debugging is part of programming. Even the most experienced developers have bugs.
The difference is not the existence of bugs – it’s how you handle them.
This is intended to help you overcome some common bugs in interviews, as something you can refer to when things don’t go as you hoped or expected.
Interviews are shorter than expected
Let’s say you’ve planned for a half an hour interview… and find both sides are running out of things to say after 10 minutes.
This is something my Software Social co-host Colleen Schnettler was running into. She’d go in with a script, all jazzed to talk… and then it would end up being over quickly.
Colleen found it was because she was getting excited that she was solving the problem they experienced.
It also commonly happens when the person is describing a problem you’re already familiar with. You may not feel the need to ask for further details.
Yet the goal is to figure out how this happens from their perspective, which can only be learned by asking them.
It takes time to adjust to asking for details about things you’re already familiar with. Most educational systems and workplaces punish this kind of question-asking, so it makes sense if it feels like a strange concept to you.
It can also be a combination setting an environment for listening and providing follow-ups.
Follow-ups don’t necessarily need to be questions. Some of the best follow-ups you can say are simple phrases.
Tools to use when this happens:
  • Descriptive statements. This can be as simple as repeating back what the person has said. For example, if someone tells you they had issues with their database, you can say “It sounds like you had issues with the database?” and wait.
  • “Can you say more about that?”
  • Simple phrases that show you’re listening, like “mhmm,” “I see”
The person seems nervous
It’s not uncommon to have someone be somewhat nervous.
Especially if you’re doing a screenshare test, someone might be worried that they’re doing things wrong.
At the beginning of every screenshare test, it’s important to say “We’re just testing the website. It’s still in early development so we know there are a lot of snags with it. The more honest you can be, the more you’ll help us find those things. You can’t do anything wrong.”
Tools to use when this happens:
  • Validating statements. “That makes sense” and “I can see why you’d do it that way” go such a long way to reassure someone who is feeling nervous about sharing their experiences.
  • Remembering to use “your most harmless voice possible.” A follow-up like “oh how interesting” can be taken very differently depending on the tone of voice. Remember to speak like you’re talking to a respected grandparent.
The person is giving short answers
You’re asking questions about process and getting very short answers back said in a hurried or clipped tone. For example:
You: So can you tell me more about how you get the data into your CRM?
Person: I-just-upload-it-and-then-its-done.
You: Oh, okay. Can you say more?
Person: Nope-that’s-it! What other questions do you have?
Tools to use when this happens:
  • Check in with them. Are they saying things quickly? This could mean they’ve had something come up, and maybe they’re pressed for time. In this case, I’d suggest checking in with them and saying “It sounds like you might be in a time crunch. Would another time be better?”
It’s okay to cut it short if they don’t have time, or find another time to talk to them. People have last-minute things come up, they have emergencies, they forget about calls. It might be annoying for you to have this happen if you’ve made time for the call, yet we have to give people grace and remember that this happens. It’s okay.
The person is being cagey
It’s worth distinguishing between someone who is pressed for time and someone who is being cagey.
You’re asking questions about process and getting very short answers back. Perhaps they’re working in answers that are evasive and encourage you to fill in the details. For example:
You: So can you tell me more about how you get the data into your CRM?
Person: Oh, I just put it in. You know how it works.
You: I’d like to hear more about how it works in your specific process.
Person: It’s pretty simple. I just do it.
Tools to use when this happens:
  • Check in with them. When this happens, I find it helpful to step back and see what their intentions are. In a very casual and curious tone, say, “You know, just to step back for a moment, I’m just curious. What made you interested in talking to me today?”
I’ve found that when I ask this person, it usually turns out that there are some misaligned expectations. Usually, it’s that the person expected to get some onboarding help or wanted to request a feature. If that happens, it’s okay to accept that this will not be an interview and pivot the conversation. (I cover those cases below.)
Very rarely, it’s turned out that the person had widely different motives. I’ve had customers ask if we were planning to take funding, are interested in exiting, or otherwise had business-level questions (rather than product-level) that weren’t exactly what we planned to discuss. Depending on your company goals and situation, this might be a welcome pivot (or not).
Person just wants to talk about feature requests
Let’s say you start out your interview like this:
You: Thank you for taking the time to talk to me. I really appreciate it. I have a handful of questions for you. Before I get started, I’m just wondering if you have any questions for me?
Them: Actually yes. Have you guys considered adding support for [X]? And I’d love to be able to do [Y]. Are those on your roadmap?
It sounds like this person has some burning questions and this was their primary motivation for taking the call.
You should respect that, and give them the space to get their questions answered. If you try to move them to the end of the call, the person may build up anxiety or frustration that their questions aren’t being answered, and it could reduce the overall quality of answers you get back.
In this case, remember the skills you’ve learned to ask questions rather than give answers.
Tools to use when this happens:
  • When someone requests a feature, you can say something like “We’ve had a few other people ask about that. I’m curious, could you tell me more about a situation when you’d use that?” and go into your normal feature request questions
  • “What do you currently use for this?”
  • “How long does this currently take you to do this?”
  • “How much do you currently pay to get this done?”
Going forward, you may also want to take a look at your recruiting copy. Having the opportunity to ask for feature requests is usually a great incentive for existing customers, yet you want to make it clear you’ll have your own questions too.
They expected onboarding
Another source of misaligned expectations comes from onboarding. I’ve generally found that this happens when the call happens too close to someone signing up.
You: You: Thank you for taking the time to talk to me. I really appreciate it. I have a handful of questions for you. Before I get started, I’m just wondering if you have any questions for me?
Person: Yeah. So how do I create an API key?
The first step here is to pull yourself back, recognize that you won’t be able to jump into your script, and answer their questions.
You can still get to their questions, but you may have to take a circuitous route there.
However, this isn’t necessarily a bad thing. Especially in the early days of a product, getting details about any issues people are having getting started, what their projects are, and why they were looking to try something new are so incredibly helpful.
Tools to use when this happens:
  • Flexibility. You hoped for one topic, and got another. You may find it helpful to create an onboarding script for yourself as a back-up in case this happens. A good place to start is “So tell me how it’s going getting started with [X].”
Like when they come in expecting to talk about feature requests, you’ll want to look at your recruiting copy and timing. From my experience, interviews are best scheduled at least a month out from when someone has started using a product. Anything sooner tends to get interpreted as onboarding.
There is a time and place for onboarding calls, and consider whether that is what you need right now. If you are having trouble activating users, you want to talk to people within the first week before they forget about it.
Multiple people are on the call from their side
Another issue can come up when it turns out that multiple team members are on a call. You may get a hunch about this in advance by seeing people confirm a forwarded event invitation, or it could be a surprise.
You: Hi, is this Owen?
Person: Yes, and I’ve also got Asuka, Marcus, and Deepak here.
Unfortunately, it’s impossible to truly interview in a group setting.
Focus groups are helpful for seeing how people behave and make decisions in a group setting. If your product is generally sold to groups, this could be a good opportunity for you to learn about their decision-making process, roles, and interactions with other decision-makers.
However, it will not give you the kind of process-level depth that you’d get from talking to one person alone. And since they’re in a group setting, people may censor themselves or be less open than they would be in a 1-to-1 setting. Also, all of the relevant stakeholders for the decision-making process may not be in the room.
It’s a tricky situation.
Tools to use when this happens:
  • Check in with them first. If this happens, check in with them immediately. Say “Wow, thank you so much everyone for taking the time to hop on this call. I had some things I was wondering about from my perspective, but let’s start out with you all. I’m interested to hear more about what led you to want to jump on today.”
Usually, it turns out that they were hoping for a training or some sort of onboarding help.
If group-level dynamics are helpful for you to understand, feel free to jump in. Just make sure you address their questions/motivations first.
They’re talking about something unrelated
Maybe you’re having the opposite problem of short answers, and you’re getting really long answers with lots of unrelated details.
I’ve had people start telling me about their cats, about their upcoming vacation, about all kinds of things.
Thankfully, that doesn’t happen often, and it’s more likely they start talking about an entirely unrelated process. Perhaps they start talking a lot about invoices when you wanted to hear about event planning.
This can be interesting, though. Because maybe it turns out your conceptualization of a problem is missing pieces, or there are adjacent steps that are far more painful than the problem you’re intending to solve.
Tools to use when this happens:
  • Check in with yourself. Is what they’re talking about completely unrelated (you asked about invoices and ten minutes later they’re talking about cats) or related in a helpful way (you asked about invoices and they’re talking about difficulties with vendor coordination)? It could be that you’ve uncovered an even bigger problem for them.
  • Polite re-steering. “Thank you for telling me that. That makes sense. I’m wondering if we could go back to something you mentioned earlier. Could you tell me more about how you use [something on-topic]?”
The worst case scenario here is that you’ve uncovered something that is important to this person that no one else has ever asked about. As a human, that’s an important moment. It builds trust and affinity. Sometimes it’s worth letting someone go on a bit about an unrelated topic to build rapport. The more you do interviews, the more you’ll get a hang for when you should dive deeper vs when you should re-steer the conversation.
They’re mad about something
This is a very rare occurrence in my experience (maybe one or two in the thousands of inteviews I’ve done across B2C and B2B), and it receives an outsized amount of anxiety among new interviewers, so I would be remiss not to mention it.
You: Hi, is this Ronald?
Person (gruff): Yeah.
You: Thank you for taking the time to talk to me. I have a few questions for you. Before we get started, do you have any questions for me?
Person: Yeah, why’d you charge my credit card $$$? What kind of service is this even?
The first thing to do when someone is mad to listen to them and let them get it out.
Next, you can determine whether this problem is solvable, and work towards a solution for this.
If you’re able to calm them down, it would be incredibly valuable to you to figure out why they signed up for your product in the first place. Do you have a landing page that sounded like it solved their problem when you solve something completely different? Are you accidentally running ads on unrelated keywords? If you can get them calm, it’s always valuable to try to figure out the source of the misunderstanding.
Whether it was an unexpected charge, a feature that didn’t work the way they wanted it to, or some other experience that didn’t go as they’d expected… if you can, try to get there.
Pivoting to their own sense of generosity sometimes helps. “This has been a really frustrating situation for you, and we’d like to use this as a chance to make sure no one else has this same experience you’ve had. Would you be able to tell me more about what you were trying to do in the first place and how you came across us?”
Tools to use when this happens:
  • Listen. Listen listen listen. Angry people first need to feel heard. Resist the urge to defend yourself or your product.
  • Establish your competence. “I can help you sort this out.” is a helpful phrase to use here.
  • Solve their problem. If you can, while they’re on the phone. Give them a refund, close their account. Angry former customers with unsolved problems are very expensive for a bootstrapped company that relies on word-of-mouth.
  • Solve your problem. Try to get to the root of where the misunderstanding came from.

Did you enjoy this issue? Yes No
Deploy Empathy
Deploy Empathy @mjwhansen

A practical guide to interviewing customers

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Created with Revue by Twitter.
440 Monticello Ave, Suite 1802 #43146, Norfolk VA 23510