Detailed explanation of file operations in server programming

Everything under the Linux system is a file. All the underlying layers are shielded through the virtual file system (VFS) mechanism. Users can operate different drivers through a unified interface. Each file needs a reference. Indicates that the file descriptor is applied at this time. The file descriptor is similar to the handle under windows. Most operations on the file are operated through this descriptor, such as read and write. For each file descriptor, the kernel uses three data structures to manage it.

(1) Each process has a record entry in the process table, and each record entry has an open file descriptor table, which can Treated as a vector, each descriptor occupies one entry. Associated with each file descriptor is:

　(a) File descriptor flag. (Currently only one file descriptor flag FD_CLOEXEC is defined)

　(b) Pointer to a file table entry.

(2) The kernel maintains a file table for all open files. Each file entry contains:

　(a) File status flag (read, write, add-write, synchronization, non-blocking, etc.).

　(b) Current file displacement. (That is, the value operated by the lseek function)

　(c) Pointer to the v-node entry of the file.

(3) Each open file (or device) has a v node structure. The v node contains pointer information about the file type and functions that perform various operations on this file. For most files, the v node also contains the i node (index node) of the file. This information is read from disk into memory when the file is opened, so all information about the file is quickly available. For example, the i-node contains the file's owner, the file's length, the device on which the file resides, a pointer to the actual data blocks used by the file on disk, and so on.

After the three-layer encapsulation of the above file system, each layer is responsible for different responsibilities, from top to bottom the first layer It is used to identify files, the second layer is used to manage process independent data, and the third layer manages file system metadata and is directly associated with a file. One advantage of this layered idea is that the upper layer can reuse the structure of the lower layer. There may be multiple file descriptor entries pointing to the same file table entry, and there may be multiple file table entries pointing to the same V node.

If two independent processes open the same file, each process that opens the file will get a file table entry, but the V node pointers of the two file table entries point to the same V node, this arrangement allows each process to have its own current displacement of the file, and supports different opening methods (O_RDONLY, O_WRONLY, ORDWR).

When a process creates a child process through fork, the file descriptors in the parent and child processes share the same file table entry, that is to say, the file descriptors of the parent and child processes pointing in the same direction. Generally, we will close the fd that we do not need after fork. For example, when the parent and child processes communicate through pipe or socketpair, they will often close the end that they do not need to read (or write). Only when there is no file descriptor referencing the current file entry, the close operation actually destroys the current file entry data structure, which is somewhat similar to the idea of ​​reference counting. This is also the difference between the close and shutdown functions in network programming. The former only truly disconnects when the last process using the socket handle is closed, while the latter directly disconnects one side of the connection without any discussion. However, in a multi-threaded environment, since the father and son threads share the address space, the file descriptors are jointly owned and there is only one copy. Therefore, you cannot close the FD that you do not need in the thread, otherwise it will cause other files that need the FD to be closed. Threads are also affected. Because the file descriptors opened in the parent and child processes share the same file table entry, in server programming of some systems, if the preforking model is used (the server pre-derives multiple child processes, and each child process listens to listenfd to accept the connection) This will lead to the occurrence of the thundering herd phenomenon. Multiple sub-processes derived from the server each call accept and are therefore put to sleep. When the first client connection arrives, although only one process obtains the connection, all processes are awakened, resulting in Performance suffers. See UNP P657.

At the same time, if exec is called after fork, all file descriptors will continue to remain open. This can be used to pass certain file descriptors to the program after exec. At the same time, the file descriptor flag FD_CLOEXEC is an option used to keep open file descriptors when closing exec.

You can also explicitly copy a file descriptor through dup or fcntl, and they point to the same file table entry. Copy the file descriptor to the specified value through dup2.

Each process has a file descriptor table, which is independent between processes. There is no direct relationship between the file descriptors between the two processes, so the file description can be passed directly within the process. descriptor, but it loses meaning if passed across processes. Unix can pass special file descriptors through sendmsg/recvmsg (see UNP section 15.7). The first three file descriptors of each process correspond to standard input, standard output, and standard error. However, there is a limit to the number of file descriptors that can be opened by a process. If there are too many open file descriptors, the problem of "Too many open files" will occur. In the network server, when accept is called through listenfd, an EMFILE error is generated. This is mainly because the file descriptor is an important resource of the system. The system resources are exhausted. The system limits the default value of the file descriptor of a single process. It is 1024, which can be viewed using the ulimit -n command. Of course, you can also increase the number of process file descriptors, but this is a temporary solution rather than a permanent solution, because when dealing with high-concurrency services, server resources are limited and resource exhaustion is inevitable.

When combined with the horizontal triggering method of epoll to listen to lisenfd connections, a large number of socket connections will fill up the TCP connection queue if not processed, and listenfd will always generate readable events. To put the server into busy waiting, Chen Shuo, the author of the C++ open source network library muduo, uses the method of preparing an idle file descriptor in advance. When an EMFILE error occurs, he first closes the idle file, obtains a file descriptor quota, and then accepts it. The file descriptor of a socket connection is then closed immediately, thus gracefully disconnecting the connection from the client, and finally reopening the idle file to fill in the "hole" in case this situation occurs again.

 1 //在程序开头先”占用”一个文件描述符 2  3 int idlefd = open("/dev/null", O_RDONLY | O_CLOEXEC); 4 ………… 5  6 //然后当出现EMFILE错误的时候处理这个错误 7  8 peerlen = sizeof(peeraddr); 9 connfd = accept4(listenfd,  (struct sockaddr*)&peeraddr, &peerlen, SOCK_NONBLOCK | SOCK_CLOEXEC);10 11 if (connfd == -1)12 {13     if (errno == EMFILE)14     {15         close(idlefd);16         idlefd = accept(listenfd, NULL, NULL);17         close(idlefd);18         idlefd = open("/dev/null", O_RDONLY | O_CLOEXEC);19         continue;20     }21     else22         ERR_EXIT("accept4");23 }

The above is the detailed content of Detailed explanation of file operations in server programming. For more information, please follow other related articles on the PHP Chinese website!