How to configure load balancing for TCP in Nginx server

1. Install nginx
1. Download nginx

# wget http://nginx.org/download/nginx-1.2.4.tar.gz

2. Download tcp module patch

# wget https://github.com/yaoweibin/nginx_tcp_proxy_module/tarball/master

Source code homepage: https ://github.com/yaoweibin/nginx_tcp_proxy_module

3. Install nginx

# tar xvf nginx-1.2.4.tar.gz
# tar xvf yaoweibin-nginx_tcp_proxy_module-v0.4-45-ga40c99a.tar.gz
# cd nginx-1.2.4
# patch -p1 < ../yaoweibin-nginx_tcp_proxy_module-a40c99a/tcp.patch
#./configure --prefix=/usr/local/nginx --with-pcre=../pcre-8.30 --add-module=../yaoweibin-nginx_tcp_proxy_module-ae321fd/
# make
# make install

2. Modify the configuration file
Modify the nginx.conf configuration file

# cd /usr/local/nginx/conf
# vim nginx.conf

worker_processes 1;
events {
worker_connections 1024;
}

tcp {
upstream mssql {
server 10.0.1.201:1433;
server 10.0.1.202:1433;
check interval=3000 rise=2 fall=5 timeout=1000;
}
server {
listen 1433;
server_name 10.0.1.212;
proxy_pass mssql;
}
}

3. Start nginx

# cd /usr/local/nginx/sbin/
# ./nginx

View 1433 port:

#lsof :1433

4. Test

# telnet 10.0.1.201 1433

5. Use sql server client tool to test

##6. The execution principle of tcp load balancing
When nginx receives a new client link from the listening port, it immediately executes the routing scheduling algorithm, obtains the specified service IP that needs to be connected, and then creates a new upstream connection. to the specified server.

tcp load balancing supports nginx’s original scheduling algorithm, including round robin (default, polling scheduling), hash (select consistent), etc. At the same time, the scheduling information data will also work with the robustness detection module to select the appropriate target upstream server for each connection. If you use the hash load balancing scheduling method, you can use $remote_addr (client IP) to achieve a simple persistent session (connections with the same client IP always fall on the same service server).

Like other upstream modules, the tcp stream module also supports custom load balancing forwarding weights (configuration "weight=2"), as well as backup and down parameters for kicking out failed upstreams. server. The max_conns parameter can limit the number of TCP connections of a server and set the appropriate configuration value according to the capacity of the server. Especially in high-concurrency scenarios, it can achieve the purpose of overload protection.

nginx monitors the client connection and the upstream connection. Once the data is received, nginx will immediately read and push it to the upstream connection without performing data detection in the tcp connection. nginx maintains a memory buffer for client and upstream data writing. If the client or server transmits a large amount of data, the buffer will increase the memory size appropriately.

When nginx receives a connection closing notification from any party, or the tcp connection is idle for longer than the proxy_timeout configured time, the connection will be closed. For TCP long connections, we should choose an appropriate proxy_timeout time, and at the same time, pay attention to the so_keepalive parameter of the monitoring socket to prevent premature disconnection.

ps: Service robustness monitoring

tcp load balancing module supports built-in robustness detection. If an upstream server refuses a tcp connection for more than the proxy_connect_timeout configured time, it will is considered to have expired. In this case, nginx immediately tries to connect to another normal server in the upstream group. Connection failure information will be recorded in the nginx error log.

If a server fails repeatedly (exceeding the parameters configured by max_fails or fail_timeout), nginx will also kick the server. 60 seconds after the server is kicked off, nginx will occasionally try to reconnect to it to check whether it is back to normal. If the server returns to normal, nginx will add it back to the upstream group and slowly increase the proportion of connection requests.

The reason for "slowly increasing" is because usually a service has "hot data", that is to say, more than 80% or more of the requests will actually be blocked in the "hot data cache". Only a small portion of the requests are actually processed. When the machine is just started, the "hot data cache" has not actually been established. At this time, a large number of requests are forwarded explosively, which is likely to cause the machine to be unable to "bear" and hang up again. Taking mysql as an example, more than 95% of our mysql queries usually fall into the memory cache, and not many queries are actually executed.

In fact, whether it is a single machine or a cluster, restarting or switching in a high concurrent request scenario will have this risk. There are two main ways to solve it:

(1) The requests gradually increase, from less to more, gradually accumulating hotspot data, and finally reaching normal service status.

(2) Prepare "commonly used" data in advance, actively "preheat" the service, and then open access to the server after the preheating is completed.

The principle of tcp load balancing is the same as that of lvs. It works at a lower level and its performance will be much higher than the original http load balancing. However, it will not be better than lvs. lvs is placed in the kernel module, while nginx works in user mode, and nginx is relatively heavy. Another point, which is very regrettable, is that this module is a paid function.

The tcp load balancing module supports built-in robustness detection. If an upstream server refuses a tcp connection for more than the proxy_connect_timeout configured time, it will be considered to have failed. In this case, nginx immediately tries to connect to another normal server in the upstream group. Connection failure information will be recorded in the nginx error log.

If a server fails repeatedly (exceeding the parameters configured by max_fails or fail_timeout), nginx will also kick the server. 60 seconds after the server is kicked off, nginx will occasionally try to reconnect to it to check whether it is back to normal. If the server returns to normal, nginx will add it back to the upstream group and slowly increase the proportion of connection requests.

The reason for "slowly increasing" is because usually a service has "hot data", that is to say, more than 80% or even more of the requests will actually be blocked in the "hot data cache" , only a small part of the requests are actually processed. When the machine is just started, the "hot data cache" has not actually been established. At this time, a large number of requests are forwarded explosively, which is likely to cause the machine to be unable to "bear" and hang up again. Taking mysql as an example, more than 95% of our mysql queries usually fall into the memory cache, and not many queries are actually executed.

In fact, whether it is a single machine or a cluster, restarting or switching in a high concurrent request scenario will have this risk. There are two main ways to solve it:

(1) The requests gradually increase, from less to more, gradually accumulating hotspot data, and finally reaching normal service status.
(2) Prepare "commonly used" data in advance, actively "preheat" the service, and then open access to the server after the preheating is completed.

The principle of tcp load balancing is the same as that of lvs. It works at a lower level and its performance will be much higher than the original http load balancing. However, it will not be better than lvs. lvs is placed in the kernel module, while nginx works in user mode, and nginx is relatively heavy. Another point, which is very regrettable, is that this module is a paid function.

The above is the detailed content of How to configure load balancing for TCP in Nginx server. For more information, please follow other related articles on the PHP Chinese website!