Every month, at least two billion people visit Amazon’s website. Many of them purchase specific products, some add items to their carts for later, while others just browse without intending to spend their money. Regardless of their actions, Amazon collects data on each of these users and learns about their habits and preferences.
When the same customer returns to the platform, it automatically recommends products that might interest them. Just based on this recommendation system, the company records as much as 35 percent of its total annual sales.
Amazon’s product recommendation model is an excellent example of using Big Data technology. By harnessing valuable data about their users, the company has successfully customized its business to meet individual customer needs, resulting in increased sales.
This scenario from Amazon, as well as numerous other global and domestic companies, is why Big Data is currently seen as a kind of “hero” that can save or improve a company’s operations. Put simply, many believe that Big Data automatically means big money, but this premise is not entirely true.
Processing large amounts of data does not guarantee profit or return on investment; on the contrary, it can be a significant cost for a company if implemented without clear goals or organization. Therefore, Big Data is not a guaranteed “golden ticket” to success, but it can become one if used wisely. We spoke with experts from IBA Group and Syntio, who explained how to leverage the benefits of Big Data technology to improve business performance.
Let’s start with the basics. Big Data involves examining vast amounts of information, such as hidden patterns, correlations, market trends, and customer preferences, with the aim of enabling a company to make informed business decisions. The importance of this technology lies in the increasing volatility of consumer preferences, particularly evident in recent years. Events such as the COVID-19 pandemic, the war in Ukraine, inflation, and others, have caused significant market uncertainty but also presented opportunities for companies looking to thrive in challenging environments.
One tool for success in such an environment is Big Data, as explained by Jason Gould, Chief Evangelist at Syntio, a company specializing in data engineering, using a recent global health crisis as an example:
Big Data can collect and analyze vast amounts of data from various sources, such as patient records, social media content, public health databases, and even wearable technology like smartwatches. For instance, a patient may indirectly mention to several doctors how they felt after recovering from COVID-19. These anecdotal pieces of information, combined with structured medical data and analyzed using Big Data techniques, could reveal trends or long-term effects of the disease that might otherwise be overlooked,” explained Gould, adding that “the same principle applies to diseases like cancer, where a patient’s journey is complex and requires a detailed understanding of treatments and their outcomes.
The potential game-changing power of Big Data was also highlighted by IBA Group, one of the largest software companies in Eastern Europe. They helped a manufacturing firm collect and consolidate all its data in a centralized location, ensuring accessibility and efficiency for analysis and informed decision-making.
By leveraging modern Big Data and cloud platforms, we seamlessly integrated data from various sources, including internal business platforms, purchased datasets, external APIs, and web resources. This integration was achieved smoothly within our well-organized Data Lake, which acts as a comprehensive repository of valuable company information, said Miloš Surla, Director of IBA Group Croatia.
Data analysts used the consolidated data to automate the intricate process of calculating the optimal price for each product in each store. The result? A significant increase in sales and revenue!
Big Data can also aid in market predictions, as explained by data scientist Marco Visibelli in Wired magazine. He gave the example of a large European car manufacturer that implemented an internal system to gain effective analytics on steel prices, with the aim of identifying the optimal time to purchase raw materials at a better cost. The system could combine several supplier databases, amounting to a total of 15 terabytes of information, leading the company to save $16 million in two years.
Of course, implementing such technology is not a process that can be completed in just a month. The longevity of the process depends on factors such as infrastructure, data volume and complexity, and the organizational culture surrounding data use. As Ana Marija Galić, Engineering Manager at Syntio, pointed out, it involves not just technology modernization but also a change in mindset.
Implementation usually involves defining goals, assessing the existing data landscape, developing a Big Data strategy, deploying appropriate technology, training employees, and continuously maintaining and updating the system. It’s an evolving, continuous process rather than a one-time event, explained Galić.
IBA Group added that the process can take years, but by adopting an iterative approach and adhering to best practices, companies can expect results within the first few months or even weeks of implementation.
From the above, it is evident that expertise is a crucial factor for the successful implementation of Big Data technology. Entering a project without adequate groundwork can result in significant losses.
There are four significant mistakes that can turn Big Data into an expense instead of a profit: implementing the technology without actual significant data to analyze, lacking a predefined application goal, overspending on unnecessary solutions, and not having a team with experience in Big Data technology.
Syntio also highlighted that higher costs can be incurred due to inefficient data queries or suboptimal use by employees, especially during the early adoption stages. Transitioning to a cloud-based system also requires a proactive data-driven culture. Without proper management, predictions of future needs and adaptability, costs can only escalate.
IBA Group also noted that one of the most common challenges is the lack of effective data management, which can result in creating a “data swamp” instead of a well-structured “data lake.”
To mitigate this risk, it is crucial to establish appropriate data zones that store purified master data or data specific to different business areas. By structuring and organizing data within these zones, companies can ensure accuracy and usability of information, minimizing the chances of financial problems. Clear understanding of data origins allows identifying and resolving potential issues and risks promptly, protecting the company from unnecessary financial losses,” explained Surla from IBA Group Croatia.
In conclusion, Big Data, in general, implies the possibility of creating profits, but the implementation of technology can also lead to significant losses if not properly prepared. Therefore, Big Data is not a magical hero but rather a tool that can help improve a company’s operations if managed by knowledgeable individuals.
