Research firm IDC held its Worldwide Big Data and Analytics 2015 Predictions web conference yesterday, looking at the long-term industry trends for the sector along with new themes that may start to appear over the coming years. The firm made a number of predictions around big data, the main one being that within the next three years all enterprises will need to invest in visual data discovery tools.

"Leading organisations are changing how they measure their operations, interactions with customers, and resource allocations," said Dan Vesset program VP for Business Analytics and Big Data research. "Faster access to more relevant data and constant experimentation is creating a further gap between leaders and the rest of the organisations. It is also creating new challenges for IT and business leaders tasked with their organisation's big data and analytics strategy and execution."

IDC’s main predictions were as follows:

• Visual data discovery tools will grow 2.5x faster than rest of the business intelligence  market, and by 2018 investing in this enabler of end user self-service will become a requirement for all enterprises.
• Over the next 5 years, spending on cloud-based Big Data analytics solutions will grow three times faster than spending for on-premise solutions, and hybrid on/off premise deployments will become a requirement.
• Shortage of skilled staff will persist, with almost 200,000 deep analytics jobs expected in the US alone by 2018.
• By 2017, unified data platform architecture will become the foundation of Big Data analytics strategy.
• Apps that incorporate advanced and predictive analytics, including machine learning, will accelerate in 2015 and will grow 65 per cent faster than apps without this functionality.
• 70 per cent of large organisations already purchase external data and 100 per cent will do so by 2019.
• More organisations will begin to monetize their data by selling them or providing value added content.
• Adoption of technology to continuously analyse streams of events will accelerate in 2015 as it is applied to IoT analytics, which is expected to grow at a 5-year CAGR of 30 per cent.
• Decision management platforms will expand at a CAGR of 60 per cent through 2019.
• Rich media analytics will at least triple in 2015 and emerge as the key driver for Big Data anlytics technology investment.
• By 2018, half of all consumers will interact with services based on cognitive computing on a regular basis.

It’s a big list, but the overall take-away is clear: big data analytics is becoming an increasingly important sector for all enterprises (and the biggest enterprises of all for big data are surely telcos). Companies therefore need to ensure they have the right resources and skills in place in order to profit from this opportunity. You have to look no further than today’s other news, regarding Apple and IBM, to see how important big data analytics for the enterprise has become.

According to an unrelated survey from consultants Nemertes, 65 per cent of organisations currently have some sort of big data initiative.

“As initiatives progress, organisations focus less on the technical hurdles, having figured many of them out and instead focus on output value,” explained Matt Craig, research lead at Nemertes. “Many solutions can work extremely well, yet if there is no way to analyse and share the information in a simple, concise manor, the value drops significantly.”

And whilst we’re on the subject, Spanish bank BBVA announced yesterday that it was buying local Big Data analytics start-up Madiva Soluciones for an undisclosed fee. There are, indeed, big hopes for big data.