what is hadoop

Hadoop is a distributed system infrastructure developed by the Apache Foundation, a software framework capable of distributed processing of large amounts of data; Hadoop processes data in a reliable, efficient, and scalable way; Users can develop distributed programs without understanding the underlying details of distribution.

#Users can easily develop and run applications that process massive amounts of data on Hadoop.

Hadoop implements a distributed file system (Hadoop Distributed File System), referred to as HDFS. HDFS is highly fault-tolerant and designed to be deployed on low-cost hardware; and it provides high throughput to access application data, making it suitable for those with large data sets. set) application. HDFS relaxes POSIX requirements and allows streaming access to data in the file system.

The core design of the Hadoop framework is: HDFS and MapReduce. HDFS provides storage for massive data, while MapReduce provides calculation for massive data.

Hadoop mainly has the following advantages:

● High reliability. Hadoop's ability to store and process data bit-by-bit is worthy of trust.

● High scalability. Hadoop distributes data and completes computing tasks among available computer clusters, which can be easily expanded to thousands of nodes.

● Efficiency. Hadoop can dynamically move data between nodes and ensure the dynamic balance of each node, so the processing speed is very fast.

● High fault tolerance. Hadoop can automatically save multiple copies of data and automatically redistribute failed tasks.

● Low cost. Compared with all-in-one machines, commercial data warehouses, and data marts such as QlikView and Yonghong Z-Suite, hadoop is open source, so the software cost of the project will be greatly reduced.

Hadoop comes with a framework written in Java language, so it is ideal to run on Linux production platforms. Applications on Hadoop can also be written in other languages, such as C.

The above is the detailed content of what is hadoop. For more information, please follow other related articles on the PHP Chinese website!