I really like the story of Kolibri games because it describes the stages of the data system evolution from the birth of the company! Not some pretty place in between. In 2016, the company was founded. In 2017, one marketing manager was writing some R scripts. In 2018, a team of three (!) built up an event hub, an SQL database, and finally Looker on top of it all. A year later with a team of five, they moved components over to databricks and snowflake.
After their acquisition, they started to roll out a more decentralized concept, a data mesh. They employed mostly full-stack teams inside business units:
- One central data engineering team (with analysts & data scientists)
- One marketing team with analysts, BI developers, and PMs
- One product team with analysts, BI developers, and PMs
They seem to enable this by using dbt on top of snowflake and an ELT pattern.
I really like the quote at the end, which is:
“Establishing a data-driven culture is quite important and sometimes even more important than building the right tech stack.”
In my opinion, it’s not just sometimes. Build a data-driven culture, decentralize responsibility for data and the right tech stack will emerge…