Thinking and practice of front-end and back-end separation based on NodeJS (1) Full-stack development_node.js

Foreword

In order to solve various problems caused by the traditional web development model, we have made many attempts, but due to the physical gap between the front/back end, the tried solutions are all similar. After learning from the painful experience, today we rethink the definition of "front-end and back-end", introduce NodeJS, which is familiar to front-end students, and try to explore a new front-end and back-end separation model.

With the rise of different terminals (Pad/Mobile/PC), the requirements for developers are getting higher and higher. Pure browser-side responsiveness can no longer meet the high requirements of user experience. We often need to develop for different terminals. Customized version. In order to improve development efficiency, the need for separation of front-end and back-end is getting more and more attention. The back-end is responsible for the business/data interface, and the front-end is responsible for the display/interaction logic. We can customize and develop multiple versions of the same data interface.

This topic has been discussed a lot recently, and some BUs in Alibaba are also making some attempts. After a long discussion, our team decided to explore a front-end and back-end separation solution based on NodeJS. During the process, we had some changing understandings and thoughts, which are recorded here. We also hope that classmates who can see it will participate in the discussion and help us improve it.

1. What is front-end and back-end separation?

During the initial discussion in the group, I found that everyone has a different understanding of the separation of front-end and back-end. In order to ensure that the discussion can be discussed in the same channel, we must first reach an agreement on what "separation of front-end and back-end" is.

The example of front-end and back-end separation that everyone agrees on is SPA (Single-page application). All the display data used is provided by the back-end through an asynchronous interface (AJAX/JSONP), and the front-end only displays it.
In a sense, SPA does achieve the separation of front and back ends, but there are two problems with this approach:

Among WEB services, SPA accounts for a small proportion. In many scenarios, there are synchronous/synchronous and asynchronous mixed modes, and SPA cannot be used as a universal solution.
In the current SPA development model, interfaces are usually provided based on presentation logic. Sometimes in order to improve efficiency, the backend will help us handle some presentation logic, which means that the backend is still involved in the work of the View layer, which is not real. Separation of front and rear ends.
The SPA-style separation of front-end and back-end is based on the physical layer (it is believed that as long as it is the client, it is the front-end, and the server-side is the back-end). This division method can no longer meet our needs for separation of the front and back ends. We believe that the division of responsibilities can be achieved Meet our current usage scenarios:

Front end: responsible for the View and Controller layers.
Backend: only responsible for the Model layer, business processing/data, etc.
The reason for this division of responsibilities will be discussed later.

2. Why should the front and back ends be separated?

Regarding this issue, Yu Bo’s article The Evolution of Web R&D Model explains it very comprehensively. Let’s give it a brief overview:

2.1 Applicable scenarios of existing development models

The several development models mentioned by Uncle Yu each have their own applicable scenarios, and no one completely replaces the other.

For example, MVC, which focuses on the backend, is very efficient in doing some synchronous display business. However, when it comes to pages that combine synchronous and asynchronous, it will be more troublesome to communicate with the backend development.
Ajax is the main SPA development model, which is more suitable for APP development scenarios, but it is only suitable for APPs because issues such as SEO are difficult to solve. For many types of systems, this development method is too heavy.
2.2 Unclear responsibilities of front-end and back-end

In a system with complex business logic, we are most afraid of maintaining code that is mixed with front-end and back-end. Because there are no constraints, each layer of M-V-C may have codes from other layers. Over time, there is no maintainability at all.
Although the separation of front-end and back-end cannot completely solve this problem, it can be greatly alleviated. Because it is physically guaranteed that you cannot do this.

2.3 Development efficiency issues

Taobao's Web is basically based on the MVC framework webx. The architecture determines that the front end can only rely on the back end.
Therefore, our development model is still that of writing a static demo on the front end and translating it into a VM template on the back end. I won’t go into the problems of this model, which has been criticized for a long time.
It is also painful to develop directly based on the back-end environment, and it is troublesome to configure, install and use. In order to solve this problem, we invented various tools, such as VMarket, but the front-end still needs to write VM and relies on back-end data, so the efficiency is still not high.
In addition, the backend cannot get rid of its strong focus on presentation and concentrate on the development of the business logic layer.

2.4 Limitations on front-end performance

If performance optimization is done only on the front end, the space is very limited, so we often need back-end cooperation to achieve sparks. However, due to the limitations of the back-end framework, it is difficult for us to use technical solutions such as Comet and Bigpipe to optimize performance.

In order to solve some of the problems mentioned above, we have made many attempts and developed various tools, but there has never been much improvement, mainly because we can only use the small space allocated to us on the backend. play. Only by truly separating the front and back ends can we completely solve the above problems.

3. How to separate the front and back ends?

How to separate the front and back ends? In fact, the answer is already in the first section:

Front end: responsible for the View and Controller layers.
Backend: responsible for the Model layer, business processing/data, etc.

Just imagine, if the front-end masters the Controller, we can do url design. We can decide to render synchronously on the server side according to the scene, or output json data based on the view layer data. We can also easily do Bigpipe, Comet, etc. based on the needs of the presentation layer. Socket and so on, it is entirely based on demand that determines how to use it.

3.1 "Full-stack" development based on NodeJS

If we want to achieve the layering in the picture above, we will definitely need a web service to help us achieve what we do on the front and back ends, so there is the "full-stack development based on NodeJS" mentioned in the title

This picture looks simple and easy to understand, but if you haven’t tried it, you will have many questions.

In SPA mode, the backend already provides the required data interface, and the view frontend can already be controlled. Why add an extra layer of NodeJS?
How is the performance of adding one more layer?
By adding one more layer, will the front-end workload increase?
Adding more layers means more risks. How to break it?
NodeJS can do everything, why do you need JAVA?
It is not easy to explain these issues clearly. Let me talk about my understanding process below.

3.2 Why add a layer of NodeJS?

At this stage, we mainly develop in the back-end MVC model. This model seriously hinders the efficiency of front-end development and prevents the back-end from focusing on business development.
The solution is to allow the front-end to control the Controller layer, but it is difficult to do so under the existing technical system, because it is impossible for all front-ends to learn Java, install a back-end development environment, and write VMs.
NodeJS can solve this problem very well. We don’t need to learn a new language, we can do what previous developers did for us, and everything seems so natural.

3.3 Performance issues

Layering involves communication between each layer, and there will definitely be a certain performance loss. However, reasonable layering can make responsibilities clear and facilitate collaboration, which will greatly improve development efficiency. The losses caused by stratification will definitely be made up for by gains in other areas.
In addition, once we decide to layer, we can minimize the loss as much as possible by optimizing communication methods and protocols.

For example:
After the Taobao Baby details page is made static, there is still a lot of information that needs to be obtained in real time, such as logistics, promotions, etc. Because this information is in different business systems, the front end needs to send 5 or 6 asynchronous requests to fill in the content.
With NodeJS, the front end can proxy these five asynchronous requests in NodeJS, and can easily make Bigpipe. This optimization can greatly improve the overall rendering efficiency.
Maybe you think it's okay to send 5 or 6 asynchronous requests on a PC, but on the wireless side, it is very expensive to establish an HTTP request on the client's mobile phone. With this optimization, the performance can be improved several times.

Taobao details: We are in the process of optimizing Taobao based on NodeJS. I will share the optimization process after it goes online.

3.4 Has the front-end workload increased?

Compared to just cutting pages/doing demos, it definitely adds a little bit, but under the current model, there are joint debugging and communication links. This process is very time-consuming, prone to bugs, and difficult to maintain.
Therefore, although the workload will increase a little, the overall development efficiency will be improved a lot.

In addition, the testing cost can be saved a lot. The previously developed interfaces were all for the presentation layer, making it difficult to write test cases. If the front-end and back-end are separated, even testing can be separated. One group of people will focus on testing the interface, and another group of people will focus on testing the UI (this part of the work can even be replaced by tools).

3.5 How to control the risks brought by adding Node layer?

With the large-scale use of Node, students from the system/operation and maintenance/security department will definitely join in the infrastructure construction. They will help us improve possible problems in each link and ensure the stability of the system.

3.6 Node can do everything, why do we need JAVA?

Our original intention is to separate the front and back ends. If we consider this issue, it would go against our original intention. Even if Node is used to replace Java, there is no way we can guarantee that various problems encountered today will not occur, such as unclear responsibilities. Our goal is to develop in layers, with professional people focusing on doing professional things. The JAVA-based infrastructure is already very powerful and stable, and is more suitable for doing the current architecture.

4. Taobao’s front-end and back-end separation based on Node

The picture above is my understanding of Taobao’s front-end and back-end separation and layering based on Node, as well as the scope of Node’s responsibilities. Brief explanation:

The top end is the server, which is what we often call the backend. For us, the backend is a collection of interfaces, and the server provides a variety of interfaces for us to use. Because there is a Node layer, there is no need to limit what form of service it is. For back-end development, they only use interface implementations that care about business code.
Below the server is the Node application.
There is a layer of Model Proxy in the Node application to communicate with the server. This layer is currently mainly to smooth out the way we call different interfaces and encapsulate some Models required by the view layer.
The Node layer can also easily realize the original vmcommon, tms (referring to Taobao content management system) and other requirements.
It is up to the developer to decide which framework to use for the Node layer. However, it is recommended to use the combination of express xTemplate. xTemplate can be shared between the front and back ends.
Everyone decides how to use Node, but the exciting thing is that we can finally use Node to easily achieve the output method we want: JSON/JSONP/RESTful/HTML/BigPipe/Comet/Socket/synchronous, asynchronous, whatever you want. How to adjust it depends entirely on your scene.
The browser layer has not changed in our architecture, and we do not hope that the introduction of Node will change your previous understanding of development in the browser.
The introduction of Node just puts the front-end control over the parts that should be controlled by the front-end.
We already have two projects under development with this model. Although it has not been launched yet, we have already tasted the benefits in terms of development efficiency and performance optimization.

5. What else do we need to do?

Integrate Node’s development process into Taobao’s existing SCM process.
Infrastructure construction, such as session, logger and other common modules.
Best Development Practices
Online success stories
Everyone’s understanding of the concept of front-end and back-end separation in Node
Safe
Performance
…
There is not much that needs to be innovated and researched technically, and there is already a lot of ready-made accumulation. In fact, the key is to clear up some processes and accumulate common solutions. I believe that with more project practice, this will gradually become a stable process.

6. "Midway"

Although the "full-stack development based on NodeJS" model is very exciting, there is still a long way to go to turn full-stack development based on Node into something stable and acceptable to everyone. We are currently working on The "Midway" project was designed to solve this problem. Although we have just started, we are getting closer and closer to our goal! !