S&P Global Market Intelligence and Immuta released a new study this week highlighting how many larger organizations are struggling to manage and use their data.
The report, conducted by 451 Research, found that 55% of respondents said the data they get for analysis is often out-of-date or stale by the time it gets to them. 451 Research surveyed 525 data leaders in the US, Canada, UK, Germany and France. All of the survey participants work for organizations that have more than 1,000 employees.
The survey’s findings represented the larger debate being had among enterprises about how to balance effective data use with data privacy and security. Of the respondents to the survey, 84% said they thought data privacy and security requirements would limit access to data at their organizations over the next 24 months.
Nearly 40% of respondents who work as data suppliers said they lack the staff or skills to handle their positions, with almost 30% citing a lack of automation as a problem.
At least 90% of those who answered the survey said data quality and trust were becoming more important than the volume or quantity of data, while the role of chief data officer is becoming increasingly prominent within organizations. A majority of respondents said the chief data officer had direct access to the CEO.
According to the survey, 60% of respondents said their organizations have a chief data officer while 40% do not. The numbers also corresponded to organization size, with larger enterprises being more likely to have a chief data officer.
“The findings are clear. As data workflows and processes have become more complex over time — and as organizational demand for data grows — there are clear points of friction in the data supply chain,” said Paige Bartley, senior analyst at 451 Research.
“Chief among them is data suppliers that have limited resources, skills shortages, and little automation being tasked with trying to deliver a steady stream of relevant data to a growing number of data consumers.”
Reliance on the cloud is also on the rise according to the survey, which found that 76% of respondents worked for organizations using cloud data technology more frequently for storage, compute and sharing over the next 24 months. For those still struggling to move to the cloud, 43% said it was because of security while 40% cited compliance issues and 35% said data privacy was a concern.
Overall, 65% of respondents said data has become more important for their own job now than it ever had been over the last 24 months.
More than 71% said the number of data consumers in their organization has steadily increased over time, with another 73% adding that more human and machine data consumers will need access to data over the next two years.
The changes to data consumption and deployment are also being affected by legislation, according to the survey, which found that 84% said their enterprise was subject to regulations like GDPR and HIPAA.
Data privacy and security are also prompting changes. More than 83% said data security rules will limit their access to data at their organization over the next two years.
Respondents also complained about the fact that data was not available in real time, expressing exasperation with ill-equipped data teams unable to deliver self-service data tools. Almost 40% said their data is only available at a point in time.
More than 62% of respondents said they used free cloud-based tools to help them handle data-focused tasks.
“Respondents from regulated organizations were also much more likely to report their organization had a cloud-first (31%) or cloud-forward (45%) adoption strategy, while respondents from non-regulated organizations were disproportionately more likely to report a cloud-conservative (46%) or cloud-skeptic (9%) strategy,” the report said. “The assumption that regulated industries or firms tend to shy away from cloud technology is outdated at best.”
Organizations are also struggling to manage data access and use, according to 65% of respondents.
Immuta CEO Matt Carroll said the disconnect between data suppliers and consumers highlights the pressing challenge for businesses and the public sector to improve speed and access to data.
“The findings make it clear that insights and business value cannot be quickly and easily generated from data unless it can be shared, modeled, and analyzed in a frictionless manner,” Carroll said.
“This report validates what our customers have experienced. The good news is, by understanding these pain points, organizations can address them and move forward to maximize the value of enterprise data and minimize risks. Investing in automation and scalability removes hurdles to cloud adoption and opens the door for more efficient data access and use to improve business outcomes.”
