What is hadoop?
(1)Hadoop is an open source framework that can write and run distributed applications to process large-scale data. It is designed for offline and large-scale data analysis. The design is not suitable for the online transaction processing mode that randomly reads and writes several records. (Recommended learning: web front-end video tutorial)
Hadoop=HDFS (file system, data storage technology related) Mapreduce (data processing), Hadoop data source can be in any form , has better performance than relational databases in processing semi-structured and unstructured data, and has more flexible processing capabilities. Regardless of any data form, it will eventually be converted into key/value, which is the basic data unit. .
Use functional expressions to Mapreduce instead of SQL. SQL is a query statement, while Mapreduce uses scripts and codes. For relational databases, Hadoop, which is accustomed to SQL, is replaced by the open source tool hive.
(2)Hadoop is a distributed computing solution.
What can hadoop do?
Hadoop is good at log analysis, and Facebook uses Hive for log analysis. In 2009, 30% of non-programmers at Facebook used HiveQL for data analysis;
Taobao Hive is also used for custom filtering in searches; Pig can also be used for advanced data processing, including discovering people you may know on Twitter and LinkedIn, which can achieve a recommendation effect similar to Amazon.com's collaborative filtering.
Taobao product recommendations are also available! In Yahoo! 40% of Hadoop jobs are run using pig, including spam identification and filtering, and user feature modeling.
Hadoop is made up of many elements.
At the bottom is the Hadoop Distributed File System (HDFS), which stores files on all storage nodes in the Hadoop cluster.
The upper layer of HDFS is the MapReduce engine, which consists of JobTrackers and TaskTrackers. Through the introduction of the core distributed file system HDFS and MapReduce processing of the Hadoop distributed computing platform, as well as the data warehouse tool Hive and the distributed database Hbase, it basically covers all the technical core of the Hadoop distributed platform.
The above is the detailed content of Hadoop is used for distributed computing, what is it. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
SublimeText3 Chinese version
Chinese version, very easy to use
mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),
DVWA
Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software
Dreamweaver Mac version
Visual web development tools
SecLists
SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.
