Analysis of the maximum number of concurrent TCP connections and identification methods for a single machine in network programming

Network programming In tcp applications, the server eavesdrops on a fixed port in advance, the client actively initiates a connection, and a tcp connection is established after a three-way handshake. What is the maximum number of concurrent TCP connections for a single machine?

How to mark a TCP connection. Before determining the maximum number of connections, let’s first take a look at how the system marks a TCP connection. The system uses a 4-tuple to uniquely identify a TCP connection: {localip, localport, remoteip, remoteport}.

Maximum number of tcp connections by client. Each time a client initiates a tcp connection request, unless the port is bound, the system will generally select an idle local port (localport). This port is exclusive and cannot be shared with other tcp connections. The data type of the TCP port is unsignedshort, so the maximum number of local ports is only 65536 on the Linux server system. Port 0 has a special meaning and cannot be used. Therefore, the maximum number of available ports is 65535. Therefore, when all are used as clients, the maximum number of TCP connections is 65535. , this connection can be connected to different serverip.

Maximum number of tcp connections for the server. The server is usually fixed on a local port and listens, waiting for the client's connection request. Without considering address reuse (the SO_REUSEADDR option of Unix), although there are multiple IPs on the server side, the local eavesdropping port is also exclusive. Therefore, there are only remoteip (that is, clientip) and remoteport (client) in the 4-tuple of the server-side tcp connection. port) is variable, so the maximum TCP connection is the number of client IP × the number of client ports. For IPV4, regardless of factors such as IP address classification and Linux software, the maximum number of TCP connections is approximately 2 to the 32nd power (number of IPs) ×2 to the 16th power (number of ports) Linux tcp connection limit , that is, the maximum number of tcp connections on a single server side is approximately 2 to the 48th power.

How many concurrent TCP connections can be on such a single server

What is given above is the theoretical maximum number of connections for a single machine. In actual environments, it is limited by machine resources, operating systems, etc., especially on the server side, and its maximum number of concurrent TCP connections is far from reaching the theoretical upper limit. The main reasons for limiting the number of connections under Unix/Linux are video memory and the number of allowed file descriptors (each TCP connection occupies a certain amount of video memory, and each socket is a file descriptor). In addition, ports below 1024 are generally reserved. port. Under the default 2.6 kernel configuration, after testing, each socket occupies between 15 and 20k. The parameters that affect the memory occupied by a socket include: rmem_maxwmem_maxtcp_rmemtcp_wmemtcp_memgrepskbuff/proc/slabinfo. On the server side, by reducing the video memory and modifying the maximum number of file descriptors and other parameters, it is no problem for the maximum number of concurrent TCP connections on a single machine to exceed 100,000. The American UrbanAirship company In the production environment, 500,000 concurrent users have been achieved. In practical applications, for large-scale network applications, the C10K issue also needs to be considered.

Let’s explain the two common senses of file handle restrictions and port restrictions

Common sense 1: File handle restrictions

Colleagues who write network server programs under Linux must know that each TCP connection occupies a file descriptor. Once this file descriptor is used up, the error returned to us when a new connection is made is "Socket/File:Can 'topensemanyfiles'.

At this time, you need to understand the operating system's limit on the maximum number of files that can be opened.

Process Limitation

Executing ulimit-n outputs 1024, indicating that a process can only open up to 1024 files, so if you use this default configuration, you can concurrently run thousands of TCP connections at most.

Temporary changes: ulimit-n1000000, and these temporary changes are only valid for the current usage environment of the currently logged in user, and will become invalid after the system is restarted or the user logs out.

Changes that become invalid after restarting (but I tested it under CentOS6.5 and found no invalidity after restarting): Edit the /etc/security/limits.conf file, and the changed content is

*softnofile1000000

*hardnofile1000000

Permanent changes: edit /etc/rc.local and add the following content after it

ulimit-SHn1000000

Global restrictions

Executing cat/proc/sys/fs/file-nr outputs 93440592026, which are: 1. The number of allocated file handles, 2. The number of allocated but unused file handles, 3. The maximum number of file handles. But in the kernel2.6 version, the value of the second item is always 0. This is not an error. It actually means that all the allocated file descriptors have been used without any waste.

We can change this value to a larger value and use root privileges to change the /etc/sysctl.conf file:

fs.file-max=1000000

net.ipv4.ip_conntrack_max=1000000

filter.ip_conntrack_max=1000000

Common knowledge 2: Is the range of port number limited?

The lower-end banners of the operating system below 1024 are reserved by the system, and 1024-65535 are used by users. Because each TCP connection occupies a port number, we can have up to more than 60,000 concurrent connections. I think there are many students who have these wrong ideas, right? (I have always felt this way in the past)

Let’s analyze it

How to mark a TCP connection: The system uses a 4-tuple to uniquely mark a TCP connection: {localip, localport, remoteip, remoteport}. Well, let's take out the explanation of accept in Chapter 4 of "UNIX Network Programming: Volume 1" to take a look at the conceptual stuff. The second parameter cliaddr represents the client's IP address and port number. As a server, we actually only use this port during bind, which shows that the port number 65535 is not a limit on the amount of concurrency.

Maximum number of TCP connections to the server: The server is usually fixed on a local port and listens, waiting for the client's connection request. Without considering address reuse (the SO_REUSEADDR option of Unix), although there are multiple IPs on the server side, the local eavesdropping port is also exclusive. Therefore, there are only remoteip (that is, clientip) and remoteport (client) in the 4-tuple of the server-side tcp connection. port) is variable, so the maximum tcp connection is the number of client ip × the number of client ports. For IPV4, regardless of incentives such as IP address classificationLinux tcp connection number limit, the maximum number of tcp connections is about 2 times 32 times Power (number of IPs) × 2 raised to the 16th power (number of ports), that is, the maximum number of TCP connections on a single machine on the server side is approximately 2 raised to the 48th power.

The above is the detailed content of Analysis of the maximum number of concurrent TCP connections and identification methods for a single machine in network programming. For more information, please follow other related articles on the PHP Chinese website!