Big data definitions and concepts

Big data refers to a collection of data that cannot be captured, managed and processed within a certain time range using conventional software tools. It requires new processing models to have stronger decision-making power, insight discovery and process optimization. Massive capabilities, high growth rates, and diverse information assets.

In "The Age of Big Data" written by Victor Meier-Schoenberg and Kenneth Cukier, big data refers to the use of random analysis methods (sampling surveys) ) such a shortcut and use all data for analysis and processing. The 5V characteristics of big data (proposed by IBM): Volume, Velocity, Variety, Value, and Veracity.

Recommended courses: Python Tutorial.

Definition

The relationship between big data and cloud computing

For "big data" (Big data) research organization Gartner gives this definition . "Big data" requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities to adapt to the massive, high growth rate and diversified information assets.

The definition given by McKinsey Global Institute is: a data collection that is so large that its acquisition, storage, management, and analysis greatly exceed the capabilities of traditional database software tools. It has massive data scale, rapid It has four major characteristics: data flow, diverse data types and low value density.

The strategic significance of big data technology lies not in mastering huge data information, but in professional processing of these meaningful data. In other words, if big data is compared to an industry, then the key to making this industry profitable is to improve the "processing capabilities" of data and achieve the "value-added" of data through "processing".

Technically, the relationship between big data and cloud computing is as inseparable as the two sides of the same coin. Big data cannot be processed by a single computer and must use a distributed architecture. Its characteristic lies in distributed data mining of massive data. But it must rely on distributed processing, distributed database and cloud storage, and virtualization technology of cloud computing.

With the advent of the cloud era, big data (Big data) has also attracted more and more attention. The analyst team believes that big data is generally used to describe the large amounts of unstructured and semi-structured data created by a company, which would take too much time and money to download to a relational database for analysis. Big data analytics is often associated with cloud computing because real-time analysis of large data sets requires frameworks like MapReduce to distribute work to tens, hundreds, or even thousands of computers.

Big data requires special techniques to efficiently handle large amounts of data over a tolerable amount of time. Technologies applicable to big data, including massively parallel processing (MPP) databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

The smallest basic unit is bit, all units are given in order: bit, Byte, KB, MB, GB, TB, PB, EB, ZB, YB, BB, NB, DB.

They are calculated according to the rate of 1024 (2 to the tenth power):

1 Byte =8 bit

1 KB = 1,024 Bytes = 8192 bit

1 MB = 1,024 KB = 1,048,576 Bytes

1 GB = 1,024 MB = 1,048,576 KB

1 TB = 1,024 GB = 1,048,576 MB

1 PB = 1,024 TB = 1,048,576 GB

1 EB = 1,024 PB = 1,048,576 TB

1 ZB = 1,024 EB = 1,048,576 PB

1 YB = 1,024 ZB = 1,048,576 EB

1 BB = 1,024 YB = 1,048,576 ZB

1 NB = 1,024 BB = 1,048,576 YB

1 DB = 1,024 NB = 1,048,576 BB

The above is the detailed content of Big data definitions and concepts. For more information, please follow other related articles on the PHP Chinese website!