Can Hadoop build a complete cloud computing platform?

Hadoop does not completely represent cloud computing. Therefore, if you want to use Hadoop to build a complete cloud computing platform, the answer is not enough. We often talk about cloud computing, but actually it uses large-scale or massive processing of computers to serve all kinds of people and all walks of life in life - so the core is "service". Regarding services, there are three commonly used ones (also the de facto standard):

Let’s talk about Hadoop. Directly stimulated by Google's three major papers, the Hadoop community emerged. Among the many open source implementations, Hadoop (the main project) can be said to be a top project among all known open source projects in cloud computing.

What are the components in cloud computing? With the current technology and scale, there is no definite conclusion. What is said today may be different tomorrow. But for Hadoop, the realized part is most of the problems that most enterprises encounter in their continuous development. Directly above:

From the perspective of the overall ecosystem, from the underlying storage, to the intermediate computing model and framework, to the upper-layer logical processing, flow, and display, all There is a corresponding open source implementation. This is the component you are talking about.

Including the new processing frameworks introduced in Hadoop 2.0 that we have seen, Spark, Storm, and YARN (replacing MR), are all improvements and implementations of the Hadoop ecosystem.

Hadoop implements a middle- and upper-layer model that performs massive computing and processing with as high availability as possible on the basis of simple hardware. Hadoop handles storage (only part of it), virtualization is not involved, and the underlying hardware Hadoop is not involved either. Whether it is Hadoop or other projects, it is just about solving all problems at the software level through vertical or horizontal expansion. is unrealistic. In terms of hardware, Hadoop only reserves or integrates hardware features in the implementation. In other words, in terms of virtualization, Hadoop is just an "APP", not the "originator" (misnomer).

So, what about a complete cloud computing platform?

According to the enterprise level, it depends on the specific enterprise direction and enterprise type, including IBM and VMware, which provide different solutions. . Generally speaking, it must be solved by single point -> cluster -> multi-layer (quasi-distributed) -> hardware -> distributed (regional distribution). As for the technology of the Hadoop system, just go directly to the picture above.

From the emergence of the concept of cloud computing to today, the information can be said to be "voluminous", but many of the information just copy and paste each other, and do not talk about the core of cloud computing. One point I want to make is that a complete cloud computing platform relies on business and provides storage and support.

If there is no business need but copying the information on the Internet or thinking that you have "utilized" certain technologies, you may just be "entertaining others and entertaining yourself". It will be easier to understand cloud computing if we look at how Internet load balancing technology develops:

Client cache –>CDN cache –>Apache&Nginx static page cache –>php and Java dynamic memory–> ;Memcache&Other Nosql–>MySQL&Oracle–>HDFS&Other Big Table

From a technical point of view, all problems are solved in a hierarchical manner (you must have written a Demo), and they are all based on different needs. By introducing different technologies, when a single layer, single point or even a cluster cannot solve the problem, new computing frameworks, cloud computing and grid computing and even the large-scale rendering required for animation are naturally introduced when needed. In short, a complete cloud computing platform is different for different companies and businesses. Take Tencent as an example, the platform has countless components. The "platform" only provides the most basic services: storage and support. Others need to be based on the business's own needs. Features are built on top of it (I believe that large companies have their own complete solutions, so I can’t say more here...). As for what level of "service" to improve, it depends on the company's business scale and needs. The supporting system, and even the company’s decision-making strategy, etc.

The above is the detailed content of Can Hadoop build a complete cloud computing platform?. For more information, please follow other related articles on the PHP Chinese website!