View profile

Methodical - Data & Optimization - Part One

It has been so long since the last time, I bet some of you even forgot this newsletter ever existed.
Methodical - Data & Optimization - Part One
By Akar Sumset • Issue #13 • View online
It has been so long since the last time, I bet some of you even forgot this newsletter ever existed. And that’s my fault. (Did this remind you Zuckerberg, too? :)) But past is called past for a reason. Let’s move forward.

5 D's of PMUX
The last time we spoke I had completed the Development part of 5 D’s. But since it’s been so long that you must be asking “What the hell was 5 D’s?”. It was the framework I’ve developed to better understand how to make use of all other methods, tools, frameworks that are available for Product Management and UX Design. 
In short, 5 D’s marries Product Management and UX Design to continuously design and deliver results generating products both for customers and companies. It has 5 components. Destination, Discovery, Definition and Design, Development, Data and Optimization. All components are guided by principles and practical methods. 
Now, I’ve founded a consultancy and training company around it called INVERTIV. Find more about 5 D’s in our intro presentation.
INVERTIV's End to End Product Management and UX Framework
Data and Optimization are impossibly wide topics. That’s why I want to set the context upfront. We’ll cover these topics for a generalist product person. For a UX Designer, UI Designer, Product Manager, Product Owner… my hope is that this will be quite helpful. But, if you are a, say, Conversion Rate Optimization expert then you might find this lacking tactical information. The reason for that is the tactical part of Data & Optimization heavily rely on tools like Google Analytics, Optimizely etc. and these tools change everyday. So, to keep this relevant to as many people as possible, we’ll focus more on fundamentals than tactics related to tools.
Deming is right. Ries is right, too. Even more.
Deming is right. Ries is right, too. Even more.
Data is Data. Not An Oracle.
Product management and UX design inherently mean uncertainty. That’s why we need principles as well as methods otherwise we can easily drift into indecisiveness. 
But it is very scary to accept the fact that we have to make decisions without enough evidence. That’s the reason why we wait too long trying to make sure and as a result, miss opportunities. 
Or we treat data like it’s an oracle that can tell us the future. This results in us trying to quantify everything and assume numbers will clearly tell us what to do. Well, they can’t. Numbers tell us a lot: what happened, when, how many times and who did it. But not the why. And that is also why they can’t tell us what to do, either. 
Deciding what to do next requires data (both qualitative and quantitative) but only to use it as an input for making a decision. We make decisions by thinking, arguing, using logic, conducting thought experiments, using mental models… analyzing the environment, policies, regulations, the culture we are in… and data is only one of those many things we use for making a decision.
To summarize, we collect and analyze data only to make decisions hoping they’ll make things better. Not for data and analysis magically make things so clear that decisions become self evident.
Step 1: Data Generation
I call this step Data Generation in order to emphasize that we don’t merely collect something already existing but do things actively to generate even more than what seems to be available in the first place. A good example for this is adding GPS to EXIF data and thus enriching the data collected data gathered through a photo.
There are two types of data: Qualitative and Quantitative. We need both to make better decisions. I want to start with qualitative. Here is why:
Not everything that can be counted counts and not everything that counts can be counted. Albert Einstein
Qualitative Data (QLT)
QLT simply means the kind of data that counting doesn’t tell us much about its essence. Complaints, user reviews, session recordings, posts on social media, messages, phone calls… all these are examples for QLT.
How To Group QLT
There are many ways to do this. I prefer the following:
1. Unrequested: Means data is generated even though we didn’t ask for it. Example: Support tickets.
2. Requested: Means data is generated because we asked for it. Example: Surveys.
3. Observed: Means we set a system in place and then observed the data generated through interactions. Example: Session recordings.
How To Manage QLT
1. Determine data sources.
2. Segment people generating data.
3.  Set up a process to generate data properly and continuously.
4. Set up a system to organize data.
5. Make data accessible, editable and open to enrichment.
Quantitative Data (QNT)
This is the kind we generally mean when we say data. It’s the kind of data it makes sense to count like sales, signups, bounce rate, page load time etc..
How To Group QNT
1. Behavioral: Means we count people’s actions. Example: Number of login actions.
2. Systemic: Means the data generated by our systems rather than people interacting with it. Example: Page load time.
3. Derivative: Means we play with data and create a new version of it. Example: Comparison of revenue on a yearly basis.
How To Manage QNT
1. Determine objectives. Both for company and users.
2. Make measurement a standard part of development.
3. Set up a system to organize data.
4. Make data accessible and open to enrichment.
Data can get messy to quickly and easily. This can be very harmful since we heavily depend on data while making decisions. We have to make sure we generate it correctly and consistently because analysis start with how we generate data. And obviously, we can’t make the right decision with wrong data.
Here are two links for more tactical tips.
QLT - Connected UX
QNT - Analytics Taxonomy
I’d appreciate if you’ve shared this newsletter with your network. New subscribers motivate me to write more and more.
Did you enjoy this issue?
Akar Sumset

Hey product person! Can't choose what to read? Can't trust what you learn? Methods (frameworks, guides, principles...) are the solution if you know when and why to use them. Unlike other newsletters, Methodical shares content in a structured way so that you know when to use what.

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue
Disbudak St. 17, Besiktas, Istanbul, Turkey