Technology- Big Data and Analytics
Business Challenge
The advancements made in the area of Information and Communication Technology (ICT) has opened up a wide spectrum of challenges and opportunities for the companies globally, irrespective of the industry they are operating in. With the deep penetration of internet and its widespread application in the form of interconnected systems and social media platforms, the amount of data generated is huge. The development and evolution of modern data analytics is a recent phenomenon, even though decision making based on data collection and statistical analyses have existed since past many centuries. Many global corporations are still lurking in the dark unable to decide on how to utilize this data to make it beneficial for the company and its business. Extracting information from these big chunks of data by eliminating the noise is still a challenge to many companies. At the first place, many of these organizations are unsure about what to retain and what to remove. The following are the major challenges faced by global corporations in handling the big data -
Identifying the right data and ways of utilizing the information
Quickness in analyzing and comprehending the data
Displaying the available information in an easy to comprehend format in multiple operating systems
Availability of data analytical skill set and ability to work on the data extraction tools
Fast evolving technology landscape and ability to adapt to the changes
Addressing the issue of data security
Role of IT in Addressing the Challenge
Data analytics is the science of examining the vast amount of high velocity data generated and extracting information out of the data by eliminating noise and making use of the information thus available for the benefit of the various associated stakeholders. Data analytics focuses on inference and helps organizations across the world to make informed decisions and draw upon conclusions. Data Analytics is enabled to a great extent by IT, but it is not the only enabler. Quite often, the data analytics requires large scale involvement of domain expertise along with IT. However, that does not belittle the role of IT in performing the big data analysis. When it comes to data analytics, one of the best and most common technologies available today is Hadoop. Hadoop is equipped to handle large volume of data at very high speeds. The technology is adopted widely by global corporations for its ability to deal with data from multiple sources including social media and automated sensors as well as for its lower costs (open source), computing power, scalability, storage flexibility, data protection and inherent anti- virus protection capability . In addition to Hadoop, the other popular open source data analysis tools include Tableau, OpenRefine, RapidMiner, Google Fusion Tables, Solver, etc. Global giants like SAS, IBM, SAP, Oracle, Microsoft, Teradata, HP etc. also have market popular big data analytics tools.
The tools enable the company to analyze the data as part of the advanced analytics disciplines such as predictive analytics, data mining, text analytics and statistical analysis. The traditional business intelligence tools and data visualization tools also plays a major role in the data analysis process. Where the traditional data warehouses and relational databases fail is the areas of handling unstructured and semi-structured data from multiple sources as well as the ability to handle large volume of data fed on a constant and continuous basis. The tools mentioned above are much advanced than the traditional Business Intelligence – Data Warehouse tools and are equipped to handle the steady stream of big data .
Forbes survey result found that 36% of the enterprises are investing majority of their time in analysis and just 13% are using big data analytics to predict outcomes . Some of the other major findings of this survey include -
73% of the companies are investing more than 20% of their overall technology budget in big data analytics and 76% of the executives expect the spending levels to increase
Big data assumes the biggest priority in Aviation (61%), Wind (45%) and manufacturing (42%) companies
65% of enterprises rely on big data analysis to monitor assets to identify operating issues
Other applications of the bid data analysis include connect equipment and collect operating data
Analyze data to provide useful insights
Perform predictions and implement operations, workforce and business decision optimizations.
As important as the technology adoption is the roles that are needed to successfully form the data analytics team. Some of the major roles include data hygienists, data explorers, business solution architects, data scientists and campaign experts .
Future Challenges
Before detailing about the future challenges for big data analytics, this article will also mention about the major trends in big data analytics. The big data analytics related technology is undergoing a fast evolution and companies are busy adopting the latest in order to stay ahead of the game. Some of the major trends to watch out for include Big data analytics in the cloud, Hadoop, Big Data Lakes, Predictive Analytics, and SQL on Hadoop, Better NoSQL, Deep Learning and In-memory analytics.
Following are some of the perceived future challenges facing big data analytics -
Accurate Financial Risk Assessment: One of the major future challenges facing the big data analytics is the integration of Predictive Analytics, Corporate Performance Management (CPM) and accurate risk modeling. The perceived benefits of this integration include real world compatibility for the CMP programs, financial forecasting with better business intelligence and a friendly user interface
Real-time unification of Business and Analytics: Analytical data should be available whenever the business demands them. Most often, the analytical data gets churned out at a much faster rate than what is actually required for the business. In rare cases, it gets late as well. For the perfect synchronization between these two factors, it is essential that the future technology has to be perfected.
Social RoI Measurement: It should be made sure that the Social RoI should be measured only in some form of financial metric like improved deal flow, cost reduction, customer acquisition etc. Developing the required tools to measure the social RoI in terms of a financial metric is one the major future challenges.
Unlocking the true potential of big data: Another major challenge for the big data analytics in the future is about managing the enhanced volume of data with the increasing penetration of social media, telecommunication networks and online retailing. With this huge influx of data, providing a personalized view for each and every individual will be a huge challenge.
Talent Scarcity: The whole world is moving towards big data analytics and empowering themselves with intelligence extracted out of the big data. The advances in the area of big data analytics are happening at a lightening pace and there is yet to have a standard platform or technology (Hadoop is closing in) for the data scientists to focus on. This throws up the challenge of availability of skilled hands to deal with the advancements in the technology- this is actually a challenge in the present day as well, but will get exaggerated in the future.
Mobile Analytics: This concept is still in paper, even though some of them have actually started piloting it. But there is complete certainty that mobile analytics is going to be one of the disruptive technologies of the future. Currently the ideas revolve around virtual extension of interfaces, but it is very early to say.
Future Role of IT
There is absolutely no ambiguity on the fact that IT has to play a major role in overcoming some of the future challenges as mentioned above. For the above mentioned challenges, many of the IT companies providing data analytics solution have already started working on addressing some of the challenges. For example, IBM, SAP and Adaptive Planning are moving ahead with addressing the problem of joining the dots between predictive analytics, CPM and user friendly interface (challenge 1). Similarly, for other challenges as well, the IT majors and the data analytics companies are progressing well with their solutions to address the challenges and make them future ready. The companies who have actively invested for these tasks include SAP, Oracle, SAS, Salesforce.com, IBM, Tableau, Informatica, Snaplogic, Adaptive Planning, TIBCO etc.
The IT and the analytics industry are also moving towards the next phase of analytics which includes the concept of Internet of Things (IoT). IoT is supposed by many as the future of big data analytics providing a holistic view of the end-to-end process, right from the design phase to the consumption by the customer, gathering data at each and every stage to draw some conclusions like customer buying behavior, preferences, supply chain efficiencies, vendor productivity etc. With the kind of advancements being achieved in the technology and data analytics, it can be safely assumed that newer technologies will evolve to handle the exponential growth of big data and perform data analytics in the most efficient, productive and useful manner.
Works Cited
Blue Hill Research. The Six Biggest Challenges Facing Big Data and Analytics. 4 October 2013. Website. 19 April 2015.
Columbus, Louis. 84% Of Enterprises See Big Data Analytics Changing Their Industries' Competitive Landscapes In The Next Year. 20 October 2014. Website. 19 April 2015.
Matt Ariker, Tim McGuire, Jesko Perry. Five Roles You Need on Your Big Data Team. 22 July 2013. Website. 19 October 2015.
Mitchell, Robert L. 8 big trends in big data analytics. 23 October 2014. Website. 19 April 2015.
Rouse, Margaret. Big Data Analytics. October 2014. Website. 19 April 2015.
SAS. Five Big Data Challenges. 2013. Website. 20 April 2015. <http://www.sas.com/resources/asset/five-big-data-challenges-article.pdf>.
—. Hadoop- What it is and Why it matters. 2015. Website. 19 April 2015.
Speigel, Eric. Six Challenges of Big Data. 26 March 2014. Website. 20 April 2015.
