Why is Dubbo rewritten in Go?

[[443126]]

Let me say a few words first

I often think about a lot of technical "why questions" when walking , sometimes I will think about a problem for a long time, and it is not finished until I can convince myself of every point of the problem. So I want to record these thoughts and form an article, which can be used as a new series. You may not be able to see the code in these articles, but you can get a glimpse of some problems that are easily overlooked, as well as the deeper "why" of the problem.

Today brings the first article, why does Dubbo need to be rewritten in Go?

Dubbo, which was born in Alibaba and was open sourced in 2011, has gone through 10 years. In 2019, it was rewritten in Go and open sourced. Now two years later, it has developed from the original V1.0.0 version to V3.0.0, with a star count of 3.8K as of now.

A colleague once asked me why an "old" project like Dubbo needs to be rewritten in Go. Does it have any practical significance?

Let me talk about some of my opinions today.

Connecting the past and the future

I think to answer this question well, we have to start with the original intention of Dubbo-go. It introduces itself like this on the github homepage:

The official Chinese translation is

Apache Dubbo Go language implementation, building a bridge between Java and Golang, and interconnecting with the gRPC/Dubbo ecosystem Interoperability leads the Java ecosystem to enjoy the technological dividends of the cloud native era.

Let me translate it in layman terms: In a company or department, some people use the Java version of Dubbo, and some people use Go. The two need to communicate, so Dubbo-Go was created to solve communication problems.

So the first question is, why does a company use Java and then Go?

Choice of programming language

For the choice of programming language, in business In the company, I think the most important thing to consider is efficiency, and other points are secondary. Because the main purpose of a commercial company is to make profits, no matter what language it is, as long as it can get equal benefits at the lowest cost, it is a good language.

Efficiency includes several aspects:

• Development efficiency. The development efficiency is high, the project can be launched as soon as possible, occupy the market, and can also save labor costs
• operation efficiency. It has high operating efficiency and can save server costs

Looking at the choices of many domestic commercial companies, such as Alibaba.

In the early days of Alibaba, PHP was used. The main consideration in choosing PHP was development efficiency. However, with the development of business, PHP's performance could not be supported, and it was necessary to change to a language with high operating efficiency.

With high operating efficiency, C/C naturally comes to mind, but the development efficiency of these two languages ​​is low. It is necessary to find a balance between development efficiency and operating efficiency, so Alibaba chose Java.

When Alibaba officially answered on Zhihu why they chose Java, they mainly considered the following: performance, simplicity and ease of learning, rich ecology, and active community

Put performance first and make it simple and easy to learn , rich ecology, and active community are actually referred to as development efficiency. It is precisely with these advantages that development efficiency is high.

When Alibaba chose Java, it developed a large number of Java middleware and cultivated a large number of Java talents. Therefore, other companies also referred to Alibaba when selecting technology, resulting in more and more The company chose Java.

The same is true for choosing Go. Some young companies may use scripting languages ​​​​such as PHP and Python in the early stage. After they develop and grow, they have to face the same problem as Alibaba: performance issues.

Go was released in 2012, and everyone had another choice. Go has high performance and is very simple and easy to use. New companies like ByteDance mainly use Go.

So on the whole, it is reasonable to choose Java or Go, and it is reasonable to exist.

Why do some companies choose Java but want to use Go?

• Compared with Java, Go language has faster startup, faster compilation speed, smaller memory footprint, and is good at high concurrency (coordination) (Processing) characteristics, so companies that already have Java will also consider Go, but currently such companies only account for a small proportion.
• Some companies do not have a mandatory technology stack, so new departments and new businesses can get rid of the constraints and choose the new language Go for development.

Summary

In summary, it is reasonable to choose Java or Go. It is also reasonable to choose both within a company. Although the proportion is not large, it is still reasonable. There is a need for Java and Go communication.

Dubbo’s victory in the RPC framework

In the early days, the company usually used a single service. When the scale reaches a certain level and the single application cannot support business development, it will choose a microservice architecture. At this time You need a useful RPC framework.

Among the RPC frameworks that can adapt to the Java language, Dubbo is the earliest open source in China and was open sourced in 2011.

Competing products similar to it such as Spring Cloud were open sourced in 2014, Weibo's Motan was open sourced in 2017, cross-language gRPC was open sourced in 2015, and Thrift2 was open sourced in 2007.

Only Thrift is earlier than it, but Thrift is just an RPC framework, while Dubbo includes out-of-the-box service management capabilities, such as service registration and discovery, load balancing, fault tolerance, dynamic configuration, etc.

It can be said that the early Java RPC framework had no choice.

Even in an era when RPC frameworks are flourishing, with so many companies using it and Alibaba’s endorsement, Dubbo still has its place.

Summary

When a company chooses the Java programming language and Dubbo framework (there are still many choices), and later wants to try Go, or some new businesses or new departments want to try Go At that time, they faced a problem, how to communicate with Java's Dubbo in Go.

Since the Dubbo protocol is a private protocol, the cost of re-implementing it in Go is still quite high. So Dubbo-Go came into being. From this perspective, Dubbo-Go still has considerable value in the communication between Java and Go.

Ending the struggle with thread pools

If you use the Dubbo framework, you often need a Dubbo gateway. For more information about the Dubbo gateway, you can refer to my article: "The Evolution of Microservice Gateways".

In this article, I introduce in detail the background, difficulties, selection, design, evolution and pitfall experience of a Dubbo gateway. I spent a lot of time introducing "what I did with the thread pool. "Struggle", in Java, threads are very precious, but if the Dubbo gateway is called synchronously, one request must occupy one thread, which results in concurrency failure, and when the thread pool is full, it will affect other requests.

So the solution is to either isolate the thread pool or change it to asynchronous calling. The isolated thread pool only solves the problem of requests not affecting each other, but the concurrency is still not improved. Changing to asynchronous calling can perfectly solve the problem, but the coding is too complicated.

Go's coroutine can just solve this problem. Go's coroutine is very lightweight and has higher scheduling efficiency, so we can write a very efficient gateway with simple code.

For example, you can intuitively feel that the performance of Nginx is obvious to all, but if you use Java to implement it, you don’t know how many machines you need to stack to achieve the performance of Nginx. However, Baidu uses Go to write the reverse proxy. Using BFE to replace Nginx shows how exaggerated its performance is.

For the introduction and principles of coroutines, you can refer to my article: "Writing Golang for a year, let’s talk about processes, threads and coroutines".

Summary

So on Dubbo gateway, Dubbo-Go also provides a new solution. Tuya Smart already has Dubbo-Go gateway for online use, and it has been open sourced as Dubbo- go-pixiu.

Paving the way for Dubbo Mesh

ServiceMesh has gradually become the next generation microservice architecture. Go is definitely a shining star language on Mesh, whether it is K8S, Docker and other cloud-native foundations. The facilities are all written in Go, and Go’s development speed and coroutine’s high concurrency capabilities make it the preferred language for Mesh.

Based on this, Dubbo's Meshization and Dubbo-Go have also paved the way for it. However, DubboMesh is still in a small-scale stage, and the complete implementation solution is not open source. From this point of view, if someone The company wants to take the road of DubboMesh, and Dubbo-Go may also be one of the points they need to consider.

Summary

Having said so much, it’s time to answer directly why Dubbo needs to be rewritten in Go. The answer to this question is still the official sentence: bridging the gap between Java and Golang. bridge. As for why we need to "build this bridge", please refer to the picture below:

The above is the detailed content of Why is Dubbo rewritten in Go?. For more information, please follow other related articles on the PHP Chinese website!