I like the way they describe the typical challenges that arise from traditional data integration stylesĀ ā¦.
āChances are relatively high that the meaning of data differs across different domains, departments, and systems. Data elements can have the same names, but their meaning and definitions differ, so we either end up creating many variations or just accepting the differences and inconsistencies. The more data we add, and the more conflicts and inconsistencies in definitions that arise, the more difficult it will be to harmonize.ā
āEnterprise data warehouses (EDWs) behave like
integration databases. They act as data stores for multiple data-consuming applications. This means that they are a point of coupling between all the applications that want to access itā
This quote I find particularly important, data warehouses introduce coupling, thatās kind of the point of it. And also something we really try to avoid in software engineering!
āABN AMRO, is an architecture which allows domains or teams to change and exchange data more independently in a federated and self-service model.ā
They really understand the idea of a platform that enables other teams to speed up and yet still keep a keen eye on governanceā¦.
āBy ensuring that everything flows through the same single logical layer, maximum transparency and increased speed of consumption is created. Within ABN AMRO, we deployed a sheer of data management capabilities in this single logical layer, for security, observability, discoverability, linage and linkage, quality monitoring, orchestration, notification, and so on.ā
I also really like how they go deep into metadata.
Governance is often an important topic in data mesh, often people feel like decentralization means losing central governance. I like a quote from ABN AMRO which basically says, data mesh is the way to ensure governance:Ā
āāWe have established a data governance body within our company, and no data is allowed to be distributed or consumed without clear ownership. So, for each data set we onboard on the central platform, we want to ensure data accountability.ā
Thatās an interesting thought to end things, data mesh means ownership, and only with ownership comes true governance.
Resources