In our diconium survey , we joined with Statista to ask around 150 CIOs and CTOs how they exploit their data potential. The result: Germany's companies still have a long way to go before everything runs smoothly regarding data. In the following interview, our expert Dennis Klemme (Director Data Architecture & Data Engineering, diconium data) explains why this is so, what potential data basically offers and how it can be exploited now:
Everyone is talking about how important data is for success. But more than every other company that was asked still hasn't fully made the most of their data. Why is that?
The reason it's hard to use data fully is because it's really complicated and there are many different types of data in a company. Trying to manage all the data from one central place doesn't work well because data is spread out across different parts of the company. This creates separate groups of data, making it tough to see the big picture. To deal with this challenge, here are some things that can help:
First, the company needs a plan for how to use data and the top bosses need to show they're serious about it. This plan should guide decisions based on data and make sure all workers understand how important data is and use it in their tasks.
Second, data stuff needs to be thought of in three ways: business stuff, technical stuff, and legal stuff. These three things are all connected and need to work together. When the company makes decisions, they need to think about all three.
Lastly, the company should share the responsibility for data among different groups and let teams from different parts of the company work together. People from different departments should team up and use their different skills to do projects that use data. It's really important for the company to have a culture where they see how valuable data is and encourage everyone to use it.
The companies surveyed expect the most significant potential for data use to be in service and customer management. Does this survey result match your experience? Where does data add the most value to your company?
The strong emphasis on service and customer management as promising areas for data use is quite understandable. In many companies, customer service is still personnel-intensive and holds considerable potential for cost savings through automation, such as chatbots.
It is important to note that the potential of data use can vary depending on the industry and company orientation. While service and customer management is undoubtedly a critical area that can benefit significantly from data use, companies should also consider other aspects such as product development, marketing strategies and internal process optimisation to generate the greatest possible added value from their data.
Every second company surveyed now uses hybrid data management. What advantage does this combination of a data lake and a classic data warehouse offer over the exclusive use of one solution?
Combining a data lake and a classic data warehouse in the form of hybrid data management offers companies various advantages over the exclusive use of just one solution. This hybrid approach allows it to keep critical and competitively relevant data locally and use cloud services' flexibility and scalability for other data.
A data lake is a centralised storage platform where organisations can collect and store large amounts of raw data in its original form. This allows organisations to cost-effectively and efficiently store valuable data that may not need to be immediately converted into a structured format. The data lake offers excellent flexibility, as different data types and sources can be integrated without prior data transformations. This promotes agility in data analysis and enables data experts to respond quickly to new issues.
On the other hand, a classic data warehouse provides a structured and optimised environment for analysing and reporting corporate data. Data is organised and prepared in a predefined schema for consistent reporting and analysis. The data warehouse provides high quality and security, which can be crucial for critical business decisions.
The hybrid data management strategy allows companies to get the best of both worlds. Critical and sensitive data that may need to be kept on-site can be managed in the data warehouse, while data that is not immediately needed or large amounts of raw data can be cost-effectively stored in the data lake and later integrated into the data warehouse as required. This flexibility provides an optimal balance between data processing efficiency, data security, and scalability.
Over half of the decision-makers surveyed use innovation and technology partnerships to accelerate their transformation processes. Why do external partners play such an essential role in digital transformation?
External partners play an essential role in digital transformation by enabling companies to benefit from external expertise, specialised resources, and best practices. There are several reasons why innovation and technology partnerships are critical to accelerating transformation processes:
-
Expertise and experience: External partners often bring expertise and experience from different industries and projects. They have specialised knowledge of the latest technologies, best practices, and innovative solutions. This expertise can help companies overcome challenges faster and make more informed decisions.
-
Access to resources: In digital transformation, resources such as highly qualified data experts, developers or data scientists can be scarce. External partners can fill the gap and provide the resources necessary promptly. This enables companies to drive transformation projects forward without delays while freeing up internal resources for core strategic activities.
-
Accelerated implementation: Companies can quickly implement complex projects by working with external partners. The partners often have proven methods and a comprehensive understanding of the implementation processes, which reduces time-to-market and promotes agility in the digital transformation.
-
Risk minimisation: Digital transformations often involve uncertainties and risks. External partners can help minimise risks and avoid mistakes through their experience and expertise. This supports companies in achieving their transformation goals more efficiently and reliably.
-
Scalability: External partners can adjust the capacity and scalability of transformation projects as needed. This is particularly important when companies face short-term challenges or demand for specific solutions increases rapidly.
The survey found that many data projects don't work well because there aren't enough people who are really good at dealing with data. We need more experts like data analysts, data engineers, and data scientists. So, what can companies do to fix this problem and make their data projects successful?
The need for more data specialists is a significant challenge for companies that want to implement data projects successfully. To counteract this shortage and lead their data projects to success, companies can take various measures:
- Retain/engage employees: Companies should actively promote and retain existing data professionals by offering attractive working conditions, development opportunities, and growth opportunities. A positive work culture and recognition of the work of data professionals are critical to retaining talent over the long term.
- Employer branding: Through targeted employer branding, companies can position themselves as attractive employers in the data field. They can highlight their values, mission, and vision for data-driven work to attract potential data specialists.
- Creating a suitable working environment: Companies should create a conducive working environment where data experts can flourish. This includes a clear data strategy that sets the framework for data-driven decisions and a positive data culture that fosters data-driven ideas and innovation.
- External service providers as support: Companies can bring in external service providers and consulting firms to bridge temporary bottlenecks or to recruit specialised experts for specific projects. These partnerships can be a valuable addition to the internal team.
- Standardisation in the data area: Higher standardisation in the data infrastructure and the tool landscape can simplify the training and collaboration of data specialists and make it more efficient.
- Predicted efficiency leaps using artificial intelligence (AI): The development of AI technologies promises efficiency gains in many areas, including data management. Companies should take advantage of the possibilities offered by AI to automate recurring tasks and relieve skilled workers.
- Evolution of Data Roles: Data roles have evolved, and organisations should create clear definitions and responsibilities for Data Engineers, Data Scientists, Data Operations, etc. This allows for targeted staff development and clear structuring of data-related work.
