Essay On Hadoop And Nosql

Providing effective and smart tools and technologies for enterprise ventures is the top most priority of the Chief Information Officers. It is completely valid and justified to invest resources in this direction. From basic utilities like reporting to advanced data extraction and performing predictive analysis based on it allows analysts and users to gain insights and create useful pattern based results which might help them in planning, estimation as well as execution.
Data forms the core of business analytics. Conventionally, CRM applications and ERP systems stored the data. But with time, the volume, speed and type of data gathered from various sources is massive. This is an amalgamation of unprocessed, semi structured, human as well as machine generated data, data from social networking sites, blogs, data generating devices etc. All this is referred to as “Big Data.” To be able to extract useful and relevant information from the chunks of data to filter out the usable data, the emerging and action oriented approache to data processing and analytics are Hadoop, which is an open source framework and NoSQL databases like Accumulo, MongoDB, Apache Cassandra etc.
Hadoop has been designed to manage exabytes of data which is distributed across various nodes in parallel. The data oriented projects can easily scale out to this approach as hadoop clusters need commodity hardware to run, which are not so expensive. The central idea is, rather than operating upon a single piece of enormous data, break it into multiple portions so that processing can be done simultaneously in parallel. Once the independent clusters are formed, data is analyzed by MapReduce framework. After this phase is complete, data is further analyzed by Data Scientists who further perform some analytics using custom built tools. Technically speaking, the components of Hadoop are:
- Hadoop Distributed File System, which forms the default layer of storage in a cluster.
- Secondary node, serves as the backup node and takes a dump of data from name node periodically.
- Job Tracker, initiates and coordinates the jobs in MapReduce
- Slave nodes, stores data and sends it to Job Tracker for processing.
Additionally, Hadoop has some complimentary sub projects too. The NoSQL databases store the results processed from MapReduce jobs. Apart from Java, Hive is another language that has been designed specifically for Hadoop.
We now have a little idea of usefulness of Hadoop and how it works. However, it is a very crude technology and is still maturing. The technologists still face issues in managing clusters and carrying out advanced analytical operations on extensive data sets. Right now, there is a dearth of developers in the industry who are able to make the best out of it.