Big Data and Cloud Computing are two of the hottest technologies today. Most organizations have adopted or have expressed a strong interest in adopting these two technologies to achieve positive business results. More and more enterprises are shifting the workloads to the cloud. The fifth annual State of the Cloud Survey conducted in January 2016 by RightScale revealed that hybrid cloud adoption has increased significantly and is “up from 58 percent to 71 percent year-over-year”.
Similar is the case with Big Data adoption. The 2016 Big Data Executive Survey conducted by NewVantage Partners revealed that 62.5% of firms now have at least one instance of Big Data in production. The report also showed that the percentage of firms having a Big Data initiative in production increased sharply from “48.2% in 2014 to 62.5% in 2015”. The expected investment in Big Data initiatives is also expected to grow nearly 5 times to 26.8% by 2017. This report also found that in 2015, 69.9% of organizations viewed Big Data as a critical part of business success.
While these two technologies have individually demonstrated their impact, gradually both these technologies are being used together for better business benefits. As Tim Byers of Motley Fool said, “… in many ways, [cloud and big data] are becoming one and the same – cloud resources are needed to support big data storage and projects, and big data is a huge business case for moving to the cloud.” To begin with we need to understand why are companies leaning-in heavily towards these two technologies.
The big deal about big data
Approximately 7.9 zettabytes of data was produced in 2015. By 2020, this number is expected to increase 40 times! That’s the sheer volume of data that we are looking at today. As the sources of structured and unstructured data are increasing, organizations are harvesting this data to view it from multiple perspectives to discover unexpected patterns that can help them discover new solutions for complex problems. It is with the help of big data that companies are looking to spark new innovations and drive new initiatives. There has been a clear shift in the conversation regarding big data as we move from asking what data to store to what can be done with that data. As the ‘volume’ of data grows, its ‘variety’ in the form of both structured and unstructured data continues to increase. At the same time, the ‘velocity’ at which it arrives is also increasing phenomenally as the number of connected devices continues to increase. In order to make use of this data tsunami, organizations have to move away from batch processing of data and leverage ‘real time analysis’. It is only when this data is processed real-time and the post-analysis insights are presented to the teams and affected immediately that organizations can derive ‘value’ from all this data.
A walk in the cloud
As the importance of data continues to grow, it has to be noted that organizations who truly want to leverage these large data sets can benefit greatly from the speed, capacity, scalability and computing power provided by the cloud. With cloud computing, organizations can analyze massive data sets without making a significant capital hardware investment to host the data internally. Additionally, the cloud offers
organizations a very flexible infrastructure where they can increase the storage capacity when needed and scale back when required and allow access to geographically dispersed teams with ease by adding some security layers for data protection.
While the coming together of these two technologies does seem like a match made in heaven, like all successful marriages these two, too need to work hard. Issues such as faster movement of data from storage to analytics, data corruption, slow computing speed, and bandwidth, though much lesser, still are some woes that need addressing. Along with this, there is a big question of data security that has been hanging like a dagger over the organizations’ head. To address this, initiatives like data encryption and permissive access and profile based usage are few things that are being done to make the cloud and big data more secure.
Having said that, it is also quite clear that we are looking at a future where big data strategies are going to revolve around the cloud…this love affair is just too strong to be kept apart.