What: I was going to provide a resource for this, but turns out, all I found I did not like. So I’ll describe it to you instead. Keep in mind, if you search for “Specification by Example”, you will usually end up with a bunch of very narrow ideas on how this works, even though it could be a very broad technique.
My perspective: Specification by Example is a fun technique you can use in the phase of Discovery Planning & Framing. I use it from time to time as an alternative to User Story Mapping. It works in environments where the user flow is already well understood, but where we still deal with a lot of complexity because of the underlying domain. It works at feature-sized ideas. Not too small, not too big.
It helps to quickly identify possible challenges like: “are we even able to provide a solution?” Is it too big? Will it provide value? And plan the work accordingly.
How it works for an (internal) data team, example:
PM: “Great, we just released our north star metric dashboard. But I think we’re still not doing enough. The other PMs aren’t using it at all, even though they are the key target market for it. I believe they need more analysis features, ways of using the north star metric to understand how they can drive it with their product…”
Sounds like there is a need for additional data products around the north star metric. The outcome is pretty clear, we want other PMs to be able to understand how they can improve their products to drive up the north star metric.
So, we can set up a quick SpecByExample/Data session. We can invite one possible PM as a domain expert, or give it a go without him using the feedback we already got. We include the engineers the PM, and if we have one, the product designer.
We draw up a table:
- One side with two columns: “Input Question & Input Data”
- Another side with two columns: “Output information & possible output decision”
Now we start iterating and filling up the table line by line. So what is a possible question a PM could have? How about “How does my product contribute to the north star metric”; What is the data this should be based on? “North Star metric, segmented by products”. Perfect. So what is the answer he would like to see? Draw up an actual (exemplary) table of the result! Then ask yourself, what is a possible decision based on this information.
If you cannot come up with a good one, then you are probably not yet at the right data product! Because every data product should help others make better decisions. So you need to drill deeper into these examples….
- “What key activities in Product X contribute how much to the NSM?”…
-“ How do these split up by customer segments? “ …
… You keep on iterating until you have a nice set of possible decisions with high value (!) and a decent set of questions together with the data. You can now pretty quickly decide which kind of medium fits best for these kinds of answers & decisions. Is it a report, dashboard, set of reports, some machine learning prediction system, or some notebook you run a longer analysis in?
And that’s all you need. I enjoy this technique for discovering things like this, well-understood feasibility and usability, but complex domains and thus hard to understand the value.