Architecture Design: Load Balancing Layer Design Plan (8) - Load Balancing Layer Summary Part 1

1. Overview

Obviously, the introduction of the previous eight articles cannot cover all the technologies of the load balancing layer, but it can be used as an introduction to tell readers an idea for learning and using load balancing technology. Although we will turn to the introduction of the "business layer" and "business communication" layer later, the introduction of the load balancing layer will not stop. In the follow-up time, we will intersperse the release of new articles on the load balancing layer, including a reintroduction of Nginx technology, new usage scenarios of HaProxy, LVS, etc.

In this article, we summarize the previous knowledge points and intentionally expand them so that readers can find new learning ideas.

2. The core idea of ​​the load balancing layer

2-1. Consistent hashing and key selection

In the article "Architecture Design: Load Balancing Layer Design Plan (2) - Nginx Installation" we Consistent hashing algorithms are introduced in detail. It also emphasized that the consistent Hash algorithm is one of the most critical algorithms in modern system architecture and is widely used in many fields such as distributed computing systems, distributed storage systems, and data analysis. For my blog post, it will be found in the load balancing layer, business communication layer, and data storage layer.

The core of the consensus algorithm is:

• Use a certain attribute of the object (this attribute can be the IP address of the server, an open port, a user name, or some kind of encrypted string. Everything you can think of has hash meaning properties), calculate an integer so that it is distributed in the range of 0 to 2 to the power of 32.
• Of course one or more attributes of a server can also be hashed and calculated, and distributed at a certain point on the ring according to the calculation, which is the blue point on the ring in the picture.
• When a processing request comes, a hash calculation is performed based on one or some attributes of the request, and demerits are distributed at a certain point on the ring based on the calculation. That is the yellow dot on the circle in the picture above.
• We agree that requests represented by the yellow dots to the left of a certain blue dot A and to the right of blue dot B will be processed by the server represented by blue dot A. This completes the solution to "who will handle it" question. Under the premise that the blue dot exists stably, requests from the same Hash convention will all fall in the same position, which ensures the stability of service processing mapping.
• When a certain blue point goes offline for some reason, the number of yellow points it affects is also limited. That is, the next client request will be processed by the server represented by other blue dots.

2-2. Polling and weighting

• Unweighted polling means that the main control node (task source point) does not consider any factors of the target node (such as CPU performance, disk performance) , network performance), the tasks are assigned in sequence according to the list order of the target nodes. This is the simplest polling and the one with the least complexity requirements for the master node to implement. My previous blog posts "Architecture Design: Load Balancing Layer Design Plan (2) - Nginx Installation" and "Architecture Design: Load Balancing Layer Design Plan (4) - LVS Principle" all introduced this simplest polling : For example, the "rr" parameter in LVS.

• The "right" in weighted polling can be regarded as the basis of "polling". "Weight" can be many possibilities, it can be the performance quantification value of the target machine, it can be a fixed number (weighted according to the fixed number), or it can be the network speed of the target node. For example, the "lc" parameter in LVS is weighted according to the number of existing "connections" on the target machine: the fewer the number of connections, the greater the chance of obtaining the processing rights for this task.

2-3. Lease and health check

The lease agreement is mainly to ensure the fact that if the server's check operation on the client fails after the "latest time", then the server will definitely log out the client's login information, and the connection information of the server on the client will also disappear (and services will no longer be provided downwards). Each time the check is successful, this "latest time" will be pushed backward.

The lease agreement is the same as the hash algorithm we mentioned. It is also the most basic design idea in system architecture design and is widely used in various types of systems. Its working principle is something that every architect needs to master. . For example: zookeeper uses this protocol to ensure that the link between the Flow node and the Leader node is normal; the distributed storage system uses this protocol to ensure that the connection between the datanode and the namenode is normal;

3. Summary of load balancing layer technology

in the previous In the blog post, I focused on Nginx, LVS, and Keepalived technologies. Due to limited time, here we will summarize several technologies mentioned in the blog post, and then expand on the DNS technology, CDN technology and hardware load technology.

3-1, Nginx technology

在负载均衡层这个大的章节中，我有三篇文章都在直接介绍Nginx的原理和使用。但是之后有朋友给我反映还想了解更多的Nginx知识，特别点名要求我再做一篇文章介绍Nginx的动态缓存。是的，我在后面的时间里是有计划介绍Nginx的动态缓存技术，还会介绍Nginx和多款主流的反向代理软件的性能对比。但这需要时间，特别是我不想去网上找一些已有的性能对比图，还是自己一边做这样的性能测试，一边做性能报告比较靠谱。

下面这些技术是我在博文中已经重点介绍过得，我们再做一下总结：

• Nginx中的连接数限制问题

重要的配置项包括：worker_processes、worker_connections。但是光是配置这些属性是不够的，最关键的是我们要打开操作系统级别的“最大文件数”限制问题。使用“ulimit -n 65535”设置本次会话的“最大文件数”限制；还要使用“vim /etc/security/limits.conf”命令，修改内核的配置信息。主要是以下两项：

<code><span>* </span>soft nofile 65535 
<span>* </span>hard nofile 65535</code>

另外，还要注意和nginx配置项中的“worker_rlimit_nofile”属性共同使用：

<code>user root root; 
worker_processes <span>4</span>; 
worker_rlimit_nofile <span>65535</span>;

<span>#error_log logs/error.log; </span><span>#error_log logs/error.log notice; </span><span>#error_log logs/error.log info;</span><span>#pid logs/nginx.pid; </span>
events { 
    use epoll; 
    worker_connections <span>65535</span>; 
}</code>

• Nginx中的Gzip技术

gzip是Nginx进行HTTP Body数据压缩的技术。下面这段Nginx配置信息是启用gzip压缩的实例：

<code><span>#开启gzip压缩服务， </span>
gzip <span><span>on</span></span>;

<span>#gzip压缩是要申请临时内存空间的，假设前提是压缩后大小是小于等于压缩前的。例如，如果原始文件大小为10K，那么它超过了8K，所以分配的内存是8 * 2 = 16K;再例如，原始文件大小为18K，很明显16K也是不够的，那么按照 8 * 2 * 2 = 32K的大小申请内存。如果没有设置，默认值是申请跟原始数据相同大小的内存空间去存储gzip压缩结果。 </span>
gzip_buffers <span>2</span><span>8</span>k;

<span>#进行压缩的原始文件的最小大小值，也就是说如果原始文件小于5K，那么就不会进行压缩了 </span>
gzip_min_length <span>5</span>K;

<span>#gzip压缩基于的http协议版本，默认就是HTTP 1.1 </span>
gzip_http_version <span>1.1</span>;

<span># gzip压缩级别1-9，级别越高压缩率越大，压缩时间也就越长CPU越高 </span>
gzip_comp_level <span>5</span>;

<span>#需要进行gzip压缩的Content-Type的Header的类型。建议js、text、css、xml、json都要进行压缩；图片就没必要了，gif、jpge文件已经压缩得很好了，就算再压，效果也不好，而且还耗费cpu。 </span>
gzip_types <span>text</span>/HTML <span>text</span>/plain <span>application</span>/x-javascript <span>text</span>/css <span>application</span>/xml;</code>

http返回数据进行压缩的功能在很多场景下都实用：

a、 如果浏览器使用的是3G/4G网络，那么流量对于用户来说就是money。

b、 压缩可节约服务器机房的对外带宽，为更多用户服务。按照目前的市场价良好的机房带宽资源的一般在200RMB/Mbps，而服务器方案的压力往往也来自于机房带宽。

c、 不是Nginx开启了gzip功能，HTTP响应的数据就一定会被压缩，除了满足Nginx设置的“需要压缩的http格式”以外，客户端（浏览器）也需要支持gzip（不然它怎么解压呢），一个好消息是，目前大多数浏览器和API都支持http压缩。

• Nginx中的rewrite（重写）技术

Nginx的强大在于其对URL请求的重写（重定位）。Nginx的rewrite功能依赖于PCRE Lib，请一定在Nginx编译安装时，安装Pcre lib。

Nginx的rewrite功能在我《架构设计：负载均衡层设计方案（3）——Nginx进阶》 这边博客中进行了讲解。

下面是一段rewrite的示例：

<code><span>#示例1：</span>
location ~* ^<span>/(.+)/</span>(.+)\.(jpg|gif|png|jpeg)<span>$ </span>{
    rewrite ^<span>/orderinfo/</span>(.+)\.(jpg|gif|png|jpeg)<span>$ </span>  /img/<span>$1</span>.<span>$2</span><span>break</span>;
    root   /cephclient;
}

<span>#location在不进行大小写区分的情况下利用正则表达式对$url进行匹配。当匹配成功后进行rewrite重定位。</span><span>#rewrite进行重写url的规则是：regex表达式第一个括号中的内容对应$1，regex表达式第二个括号中的内容对应$2，以此类推。</span><span>#这样重定位的意义就很明确了：将任何目录下的文件名重定位到img目录下的对应文件名，</span><span>#并且马上在这个location中（注意是Nginx，而不是客户端）执行这个重写后的URL定位。</span><span>#示例2：</span>
server {
    。。。。
    。。。。
    location ~* ^<span>/orderinfo/</span>(.+)\.(jpg|gif|png|jpeg)<span>$ </span>{
        rewrite ^<span>/orderinfo/</span>(.+)\.(.+)<span>$ </span> /img/<span>$1</span>.<span>$2</span>   last;
    }

    location / {
        root   /cephclient;
    }
}

<span>#在server中，有两个location位置，当url需要访问orderinfo目录下的某一个图片时，rewrite将重写这个url，</span><span>#并且重新带入这个url到server执行，这样“location /”这个location就会执行了，并找到图片存储的目录。</span></code>

• Nginx的图片处理模块

http_image_filter_module 是nginx的图片处理模块，是使用nginx进行静态资源和动态资源分开管理的关键引用技术。通过这个模块可以对静态资源进行缩放、旋转、验证。

需要注意的是，http_image_filter_module模块所处理的缩率图片是不进行保存的，完全使用节点的CPU性能进行计算，使用节点的内存进行临时存储。所以如果要使用http_image_filter_module进行图片处理，一定要根据客户端的请求规模进行nginx节点的调整。并且当站点的PV达到一定的规模时，一定要使用CDN技术进行访问加速、对图片的访问处理手段进行规划。

由于我们在之前涉及Nginx的文章中，并没有详细讲解Nginx的图片处理模块，只是说了要进行介绍，所以这里我给出一个较为详细的安装和配置示例：

nginx的http_image_filter_module模块由GD library进行支持，所以要使用这个图片处理模块，就必须进行第三方依赖包的安装：

<code>yum <span>install</span> gd-devel</code>

然后，Nginx要进行重新编译：

<code>configure <span>--</span><span>with</span><span>-http_image_filter_module</span>
make <span>&&</span> make install</code>

使用图片处理模块的配置示例：

<code>location ~* /(.+)_(\d+)_(\d+)\.(jpg|gif|png|ioc|jpeg)$ {
    <span>set</span><span>$h</span><span>$3</span>;
    <span>set</span><span>$w</span><span>$2</span>;
    rewrite /(.+)_(\d+)_(\d+)\.(jpg|gif|png|ioc|jpeg)$ /<span>$1</span>.<span>$4</span><span>break</span>;

    image_filter resize <span>$w</span><span>$h</span>;
    image_filter_buffer <span>2</span>M;
}</code>

其中关于正则表达式的语法和已经介绍过的rewrite的语法就不再进行介绍了，主要看http_image_filter_module相关的属性设置：

image_filter test：测试图片文件合法性
image_filter rotate：进行图片旋转，只能按照90 | 180 | 270进行旋转
image_filter size：返回图片的JSON数据
image_filter resize width height：按比例进行图片的等比例缩小，注意，是只能缩小，第二缩小是等比例的。
image_filter_buffer：限制图片最大读取大小，没有设置就是1M；根据不同的系统最好设置为2M—3M
image_filter_jpeg_quality：设置jpeg图片的压缩比例（1-99，越高越好）
image_filter_transparency：禁用gif和png图片的透明度。

• 和Nginx类似的其他技术/软件

目前行业内也有很多与Nginx解决同类问题的软件，他们分别是Apache基金会的 Apache HTTP Server、淘宝开源的Tengine、Haproxy、包括Windows 下运行的IIS，也支持反向代理 。

这里笔者再次重点提到Tengine，建议各位读者有时间的时候可以使用一下，这个对Nginx进行了深度再开发的软件。

3-2、LVS技术

LVS is the abbreviation of Linux Virtual Server, which means Linux virtual server. It is a virtual server cluster system. This project was established in May 1998 by Dr. Zhang Wensong.

LVS cluster adopts IP load balancing technology and content-based request distribution technology. The scheduler has a very good throughput rate and evenly transfers requests to different servers for execution, and the scheduler automatically shields server failures, thus forming a group of servers into a high-performance, highly available virtual server. The structure of the entire server cluster is transparent to the client, and there is no need to modify the client and server programs.

In my series of articles, "Architecture Design: Load Balancing Layer Design Plan (4) - LVS Principle", "Architecture Design: Load Balancing Layer Design Plan (5) - LVS Single Node Installation", "Load Balancing Layer Design Plan (7) - LVS + Keepalived + Nginx Installation and Configuration" all involve the explanation of LVS.

Here we summarize the three working modes in LVS:

3-2-1, NAT mode

NAT mode is a datagram received by the LVS Master service node and then transferred to the lower Real Server node , when the Real Server completes processing, it sends it back to the LVS Master node and then forwards it out by the LVS Master node. The LVS management program IPVSADMIN is responsible for binding forwarding rules and completing the rewriting of attributes in IP data packets and TCP data packets.

The advantages of LVS-NAT mode are:

• Simple configuration and management. The working mode of LVS-NAT is the easiest to understand, the easiest to configure, and the easiest to manage among the three working modes of LVS.

• Save external network IP resources. The number of IPs allocated to users in general computer rooms is limited, especially when the number of racks you purchase is not large. LVS-NAT works by encapsulating your system architecture in the LAN. LVS only needs an external network address or external network address mapping to achieve access.

• The system architecture is relatively closed. In an intranet environment, we do not have high requirements for firewall settings, and it is relatively easy to operate and maintain physical servers. You can set requests from the external network to be filtered by the firewall, while requests from the internal network are open to access.

• In addition, the Real Server does not care about the authenticity of the rewritten data message transferred to the Real Server. As long as the TCP verification and IP verification can pass, the Real Server can process it. Therefore, the Real Server in LVS-NAT working mode can be any operating system, as long as it supports the TCP/IP protocol.

3-2-2, DR mode

The DR working mode of LVS is the most commonly used working mode in the production environment at present, and there is the most information on the Internet. Some articles still explain the DR working mode. More thoroughly:

The advantages of LVS-DR mode are:

• solves the forwarding bottleneck problem in LVS-NAT working mode and can support larger load balancing scenarios.

• It consumes more external IP resources. The external IP resources of the computer room are limited. If this problem does exist in the formal production environment, a mixture of LVS-NAT and LVS-DR can be used to alleviate it.

LVS-DR of course also has shortcomings:

• The configuration work is a little more troublesome than the LVS-NAT method. You need to at least understand the basic working methods of the LVS-DR mode to better guide yourself in the LVS-DR mode. Troubleshooting of problems during configuration and operation.

• Due to the message rewriting rules of LVS-DR mode, the LVS node and the Real Server node must be in the same network segment, because Layer 2 switching cannot cross subnets. But this problem actually has no essential limit for most system architecture solutions.

3-2-3, TUN mode

The working principles of LVS-DR mode and LVS-TUN mode are completely different, and the working scenarios are completely different. DR is based on data packet rewriting, and TUN mode is based on IP tunneling. The latter is the re-encapsulation of data packets:

IPIP tunnel. Encapsulate a complete IP message into the data part of another new IP message and transmit it to the designated location through the router. In this process, the router does not care about the content of the original protocol being encapsulated. After arriving at the destination, the destination relies on its own computing power and support for the IPIP tunnel protocol to open the encapsulation protocol and obtain the original protocol:

It can be said that the LVS-TUN method basically has the advantages of LVS-DR. On this basis, it also supports cross-subnet penetration.

3-3, CDN technology

CDN technology Content Delivery Network: content distribution network. Why sometimes our access to video resources and picture resources on the Internet is slow or even fails. One important reason is that the physical location of the resource is too far from the client, and there may be a layer 4 NAT device in it (equivalent to using Netcom lines to access resources on the telecommunications server).

Let’s imagine if the resources we want to access are placed on a service closest to our client (for example, the resources accessed by the client in Guangzhou are in the computer room in Guangzhou). So does this solve the problem (this point is called an "edge node"). This is the problem that the CDN network solves, as shown in the figure below:

Currently, CDN services do not require us to develop. There are many companies on the market that provide free/paid CDN services (please enter directly on Google or Baidu : CDN, there will be a lot of "promotion" information, no advertising in my blog ^_^). Of course, if you want to build a CDN network by yourself, you can refer to the following technical solutions:

Squid: Squid is a software that caches Internet data. It receives download requests from users and automatically processes the downloaded data. At present, the networks of many domestic CDN service providers are built based on Squid

Using Nginx's proxy_cache to build: the rewrite technology in Nginx can actually realize URL request rewriting and request forwarding. The proxy_cache component in Nginx can save the source data requested from the remote end locally, thereby realizing the construction of a CDN network.

Write it yourself: There is no particularly complex technical threshold for CDN networks. If you have special needs, you can write one yourself. Of course, the CDN network introduced in the above picture belongs to the first generation CDN network. Adding the second/third generation P2P technology to the CDN principle can form a second generation CDN network: as shown in the figure below:

The third generation of P2P technology, also known as hybrid P2P technology, is mainly to solve the processing pressure of metadata servers and accelerate the localization speed of resources. Regarding P2P technology, after I finish talking about "Business System Design" and "Business Communication System Design", I will make a new topic to introduce it. In addition, I would like to mention that YouTube’s P2P network is built by itself.

4. Later introduction

There is too much to summarize, so I decided to open another article "Architecture Design: Load Balancing Layer Design Plan (9) - Load Balancing Layer Summary Part 2" to continue the load Summary of balancing layer techniques. We will summarize Keepalived, DNS technology, hardware load, and introduce broader load balancing technology to you.

Copyright Statement: This article is an original article by the blogger and may not be reproduced without the blogger's permission.

The above introduces the architectural design: load balancing layer design plan (8) - the load balancing layer summary, including the above aspects, I hope it will be helpful to friends who are interested in PHP tutorials.