Daemons:
Daemons are resident programs which constitute to hadoop running. In computing terms it is a process that runs in the background.
Hadoop has five such daemons. They are 1.NameNode,
2.Secondary NameNode
3.DataNode
4.JobTracker
5.TaskTracker.
Each daemons runs separately in its own JVM. We discuss about these daemons in this post as they are associated with HDFS and MapReduce.
Hadoop is not a database: Hadoop is an distributed file system and not a database. It simply uses the filesystem provided by Linux to store data
What is Hadoop?
Apache Hadoop is an open-source java framework that is used to store,analyze and process Big data in a distributed environment across clusters of computers. It is used by Google, Facebook, Yahoo, Youtube, Twitter, LinkedIn and many more. Hadoop is top level apache project developed by Doug Cutting. It works on distributed system. It is inspired by google mapreduce algorithm for running distributed application. Hadoop could be good if you have a lot of data you do not know what to do and you do not want to loose it.
Hadoop is initiated and led by YAHOO. hadoop is written in java programming language with some code in C and some commands in shell-scripts. Companies that offer hadoop services are - IBM, Amazon Web Services, Cloudera, Microsoft Azure, Dell.
3V's of Big Data:
1.Volume:
The amount of data which we deal with is of very large size of Peta bytes.
2.Variety:
Data Comes in all type of format.(Text,audio,image,video).
3.Velocity:
The data is generating at a very fast rate. Velocity is the measure of how fast the data is coming in. For time critical applications faster processing is very important. Exm- Share marketing, Video streaming
Application of Big data:
Some applications of big data are as follows:
1. Healthcare Providers:
The big data is use in the field of medicine and healthcare. It is a great help for even physicians to keep track of all the patients history.
2. Google Search:
When we search anything google makes use of data science algorithms to deliver the best result for our searched query in fraction of seconds. Next time, when we search anything based on our previous search google gives us some recommendations.
3. Education:
Big data has great influence in the education world too. Today almost every course of learning is present online. Along with the online learning, there are many examples of the use of big data in the education industry.
4. Recommender Systems:
A lot of companies used recommender system to promote their products/suggestions in accordance with user’s interest and relevance of information. Internet giants like Amazon, Google, Flipkart and many more uses this system to improve user experience. The recommendations are made based on previous search results for a user.
Example- When we search any product on amazon, we always get recommendations about similar product. They not only help you to find relevant products from billions of products available with them, but also adds a lot to the user experience.
5. Banking Zones and Fraud Detection:
Big data is hugely used in the fraud detection in the banking sectors. In banking sector, it finds out all the mischief tasks done. It detects the misuse of credit and debit cards, business clarity, public analytics for business and IT strategy fulfillment analytics.
6. Super Market:
Big data analysis is also used in super market for market basket analysis. Market Basket Analysis is one of the most common and useful types of data analysis for marketing and retailing. The purpose of market basket analysis is to determine what products customers purchase together. A store could use this information to place products frequently sold together into the same area.
Example- People who buy bread also buy butter or people who buy shampoo might also buy conditioner.
7. Security Enforcement:
Big data is applied for improving national security enforcement. These techniques are used to detect and prevent cyber attack. Police force use big data tools to catch criminals and even predict criminal activity.
Advantages of Big Data:
1.Access to large volume of data.
2.Allows businesses to develop more effective strategies towards competitors in less time.
3.Improve Decision making capabilities.
4.Can analyse data easily.
5.Allows businesses to detect errors and fraud quickly.
6.Offers businesses a chance to improve profits and customer service.
7.Integration of both structured and unstructured data.
8.Implementing new strategies and improve service dramatically.
Ther are 3 types of Big Data
1.Structured Data:
The data which can be stored and processed in table(rows and column) format is called as a structured data. Structured data is relatively simple to enter,store and analyze.
Exm- Relational database management system.
2.Unstructured Data:
The data with unknown form or structure is called as unstructured data. The fundamental challenge of unstructured data sources is that they are difficult for nontechnical users and data analysts to understand and process.
Exm- Text files,images,videos,email,customer service interactions,webpages,PDF files,PPT,social media data etc.
3.Semi-structured Data
Semi-structured data is data that is neither raw data nor organized in a rational model like a table. It may organized in tree pattern which is easier to analyze in some case. XML and JSON documents are semi structured documents.
