Interview with Kunal Agarwal and Nick Jewell

Many organisations have diverted to new trends and technologies in order to aid faster decision-making amid a rapidly transforming business environment, and one area in which this is particularly palpable is Big Data. With this in mind, we’ve spoken to two industry experts to get the skinny on why leveraging Big Data should be the main priority for businesses of all sizes in the next twelve months, as well as how best to implement a strategy that incorporates the concept in 2019 and beyond.

Typically in January, people make personal resolutions and commitments to being better, achieving more and moving faster. The same should be said for most businesses, too: each new year should bring firm determination to become more agile, resilient and profitable.

Many organisations have diverted to new trends and technologies in order to aid faster decision-making amid a rapidly transforming business environment, and one area in which this is particularly palpable is Big Data.

Big Data helps organisations harness their data and use it to identify new opportunities or fixable omissions in business practices. By leveraging this information businesses can benefit from smarter and more strategic decisions, more efficient operations, happier customers, and higher profits. Key to this is the integration of big data analytics. Software that automates this process is increasingly allowing businesses to access this information and act upon it, effectively in real time.

With this in mind, we’ve spoken to two industry experts to get the skinny on why leveraging Big Data should be the main priority for businesses of all sizes in the next twelve months, as well as how best to implement a strategy that incorporates the concept in 2019 and beyond:

Realising the concept and making it happen

It could be argued that the implementation of leveraging Big Data learnings is still in its infancy across the board. “Companies of all sizes recognise the tremendous potential for data, but even as we move into 2019, many struggle to turn data into actionable insights,” comments Nick Jewell, Director of Product Strategy at data science and analytics company Alteryx. Indeed, while 2018 saw a big jump in the number of companies adopting a Big Data-led strategy, there is still a long way to go: Dresner Advisory Services’ 2018 Big Data Analytics Market Study found that, while big data adoption in enterprises has soared since 2015 when it was at 17 per cent, a percentage of 59 per cent in the past year proves there is a long way to go until all businesses are realising and acting upon the benefits.

Kunal Agarwal, CEO at Unravel Data, would suggest that the slow rate of adoption is due, at least in part, to the growing and conflicting areas of demand on the IT team: “In order to achieve a fully holistic approach to delivery underpinned by big data insights, there needs to be a balance of innovation and control, testing and production, efficiency and effectiveness. Finding that balance will require enterprises to take some calculated risks, but this can be done without live production suffering as automation and intelligence supports the DevOps team in their task of making the magic happen.”

SEE ALSO: Can blockchain really make big data better?

Empowering all staff to become citizen data scientists

As is the case in many areas of IT, organisations looking to leverage insights from Big Data are lacking the personnel resources and expertise required, and this is an area where companies would do well to focus their attention. “It’s a widely acknowledged truth that the talent gap within the DevOps and big data space is becoming quite a challenge. In a study conducted with Sapio Research, we found that 36 percent of enterprise business and IT cited talent scarcity a huge pain point,” commented Kunal. “The skills shortage is no new phenomena in this area of IT, nor the wider tech space; as a relatively new IT concept, it takes time to educate, train and empower staff to fully embrace it.”

Nick Jewell from Alteryx would argue that enabling employees across the board – and not just from the IT department – is crucial to harnessing the true power of these unstructured data sets. “Organisations need to empower every data worker, regardless of technical skill, to perform business process automation. Self-service analytics, where business employees can collect data from disparate sources, transform and analyse that data within a code-free environment, and determine their preferred end-state, is the way ahead.

“Traditional data cataloguing solutions do the heavy lifting in IT, but organisations now see the advantage in helping analysts from different silos of the business come together to interact with both the data and each other at the level of business understanding. This type of social collaboration, involving the business user, is critical for advancing data strategies and will help organisations look beyond corporate data assets to second- and third-party sources that can enrich the respective case and provide better outcomes.”

Asking the right questions

Even if an organisation is already actively implementing tools to make big data a valuable company asset, the journey doesn’t stop there. Businesses in 2019 should ultimately be all-questioning about their data volumes and the ways they are being utilised in order to stay competitive and resilient. “Enterprises must find a way to stitch the fabric of the data stack together to get fast, usable insights into the operations powering their business intelligence, customer service, and forecasting applications, comments Kunal from Unravel. “Efficiency and effectiveness, however the business measures the KPIs, must be tightly controlled or else the whole stack will rapidly spiral out of control in 2019.”

What form these questions take may be subjective to companies in different industries, but Nick from Alteryx provides a good template: “What data is available and is it accessible? What data do I need to acquire externally to drive competitive differentiation? Is my data available for machine learning? And, perhaps most importantly, how do we upskill our line-of-business staff, what requires pure data science know-how and what can the IT organisation manage”

SEE ALSO: What role will Java play in the future of Big Data and IoT?

Bring it all together

A lot of corporate big data activity has happened organically. Big data stacks have grown unwieldy, and there isn’t always the skilled talent for either DevOps or deep statistical analysis.

Some would suggest automation is the right thing to help manage the nuts and bolts of the software stack and ensure that the analytical talent is focused on maximising use of these large enterprise investments. With greater efficiency in this area the engineering and IT team can make strategic use of their resources.

It can be expected too that the right analytical platforms can help the latter part of the puzzle: ensuring that more parts of the business can interact and use their data resources without causing unnecessary strain on the IT function to manage what users can learn to do for themselves in a self-service environment.

In essence, 2019’s data delivery will arguably be built on leveraging the twin powers of automation and human activity, and in hand, helping humans do more with data.