What is Data Mesh

First off, let us set the scene. In our world today, when researching Data Mesh, you might have come across phrases like “a sociotechnical approach to build a decentralized data architecture”, data domains, and Data As A Product. The trouble is, that all these phrases are so high level they don’t really give you a recipe to follow, so that when you’re finished you can say, “we’ve done it, now my company has data mesh.” The concepts are simple to understand; decentralised, data as a product, and data owners are the ones that know the data best. However, understanding the actions needed to achieve these concepts is harder. Maybe, if we look at the desired outcome and work backwards, we can understand those steps.

What Do You Get Out of Data Mesh?

The goal of most companies is to make money and from that, a healthy profit. By providing customers with products and services that they want at the right quality and price point, we can do that. Sounds simple right? Let us throw into the mix, the changing needs of the customer and disrupting competitors in a rapidly changing global economy. We can expand the goal to be – meeting the customers in a way that protects or increases our revenue, whilst being able to quickly adapt to disruption, and minimising the cost of operations. This is much more realistic to how most companies work to fulfil their goals, they want to be agile and create value. This is where our agility comes in, as after all, being able to quickly adapt is what agility really is. Having business agility means meeting the customer needs quicker, creating happier customers, but it also means less disruption by competitors. The great thing is, it’s much cheaper for you as a company, as less time and effort needs to go into changing existing solutions to meet those new challenges, and so the cost of operations reduces. This is the holy grail: happy customers, business agility and cheaper operations, and who doesn’t want that?

How Does Syntio Envision Data Mesh

When you read about Data Mesh you see a conceptualised and very high-level description of a utopian world where all data is shared, and companies can enjoy the benefits of bringing that data together to gain insights or improve operational effectiveness. How though, do we translate that sentiment into something operational and actionable? In a perfect world we would have cross functional teams, publishing data as a product, and an updated data catalogue with information about the data, such as its quality, security and privacy, all things that need to be taken care of, and then finally, where to get the data. The trouble is that many companies will look at data mesh as a technology problem when it’s all about mindset and approach.

Our solution to this is to focus on the process, governance, and mindset from day one. We then can create patterns and reusable assets to compliment how the company will create their data mesh solution. Having the technology part of the solution be out of the box and easily usable means we can focus on the parts that will make the real difference.

How do we get it rolled out and really make that impact?

With our clients we typically work with enablement teams and patterns of usage and supporting technology. We want the business to be able to oversee their destiny. The customer creates cross functional teams that include all the roles needed to capture, transform, govern, and publish the data for each data creator. Data-as-a-Product also changes the responsibility model. Source data providers become responsible for the quality of the data instead of centralized Data teams, and this is a fundamental shift that massively improves data quality. Keep in mind that a data consumer can also be a data publisher if they are creating new data from the data they are consuming.

Let’s have a look at the challenges here and how we solve them. Mindset is obviously the hardest, and I’m not going to pull the wool over your eyes… this is really hard, the organisation has to want to change to get the best benefit. You could equate this to an agile transformation. Many companies went agile – small A – by adopting an Agile framework – large A – but they did this without the companywide mindset change required to really have the best impact. Unfortunately, this will be the same for data mesh, it will be seen as a technology problem. This is the challenge we spend the most time on, though some clients come to us with this approach already in hand. The next challenge- you want to get the benefits without spending a year or more making an infrastructure to support the needed changes to ways of working with data. This again is a challenge and will become more of a challenge over time. Much like with Agile, companies will be inundated with offers for Data Mesh platforms and products, most of them will be tech solutions and centrally managed. This is really an antipattern. Teams need to be able to achieve their goals, adhere to standards and the correct governance, whilst also being able to publish their data. Ideally this needs to be self-service. The next challenge will be – unless you already have a super modern architecture- you’ll have some systems, possibly legacy, that really don’t want to easily give up their data.

The Future

Ok, we have discussed the problems, but really, we’re here to give our view on the fast track to data mesh. Our approach is to use our Dataphos platform, this gives us a decoupled real-time data platform. It can be simply and easily used in each domain, giving great flexibility and if needed, real-time data for consumers. The addition of a data catalogue is basically all you need: it’s low code, simple to operate, and best of all, free to use. This allows you to focus on the mindset and organisational changes needed, as the technology problem has already been solved.

We see data mesh in the same way as agile. For years we have tried to decouple monolithic systems to remove dependencies between systems to reduce the management overhead. The introduction of DevOps teams – further reduces unnecessary dependency management. To us, Data Mesh is the agile transformation of data, we want to decouple data from the systems that create it, organisation’s monolithic systems are now the data platforms many have relied on for years. Data needs to go through the same revolution, it is your gold, it has intrinsic value. Going through this process and continually improving how data flows in your organisation, using DataOps teams, will mean that your operations are cheaper, time to value is vastly improved, and best of all – having business agility means you’re always ready to take on the next competitor.