Linux performance tuning~

The Linux operating system is an open source product, and it is also a practice and application platform for open source software. Under this platform, there are countless open source software supports, such as apache, tomcat, mysql, php, etc. The biggest concept of open source software is freedom and openness. Therefore, as an open source platform, Linux's goal is to achieve optimal application performance at the lowest cost through the support of these open source software. When it comes to performance issues, what is mainly achieved is the best combination of the Linux operating system and applications.

1. Overview of performance issues

System performance refers to the effectiveness, stability and response speed of the operating system in completing tasks. Linux system administrators may often encounter problems such as system instability and slow response speed. For example, when building a web service on Linux, web pages often cannot be opened and the opening speed is slow. When encountering these problems, some people will Complaining that the Linux system is not good is actually superficial. When the operating system completes a task, it is closely related to the system's own settings, network topology, routing equipment, routing policies, access equipment, physical lines and other aspects. Problems in any link will affect the performance of the entire system. Therefore, when a problem occurs in a Linux application, a comprehensive investigation should be carried out from the application program, operating system, server hardware, network environment, etc., to locate the part where the problem occurs, and then solve it centrally.

In terms of applications, operating systems, server hardware, network environment, etc., the two aspects that have the greatest impact on performance are the application program and the operating system, because problems in these two aspects are difficult to detect and are highly concealed. As long as there is a problem with hardware or network, it can usually be located immediately. The following mainly explains the performance tuning ideas for the operating system. Specific issues in the application need to be dealt with in detail.

The following introduces the general ideas and methods of optimizing Linux from four aspects: factors affecting Linux performance, people involved in analyzing performance, system performance optimization tools, and system performance evaluation standards.

2. Factors affecting Linux performance

2.1 System Hardware Resources

1. CPU

CPU is the foundation for the stable operation of the operating system. The speed and performance of the CPU determine to a large extent the overall performance of the system. Therefore, the more CPUs and the higher the main frequency, the better the server performance will be. But that's not entirely true.

Currently, most CPUs can only run one thread at the same time. Hyper-threaded processors can run multiple threads at the same time. Therefore, the hyper-threading feature of the processor can be used to improve system performance. Under Linux systems, Hyper-Threading is only supported when running SMP kernels, but the more CPUs you install, the less performance gain you get from Hyper-Threading. In addition, the Linux kernel will recognize multi-core processors as multiple separate CPUs. For example, two 4-core CPUs will be regarded as 8 single-core CPUs under the Lnux system. However, from a performance perspective, two 4-core CPUs and 8 single-core CPUs are not completely equivalent. According to test conclusions drawn by authoritative departments, the overall performance of the former is 25% to 30% lower than the latter.

Applications that may experience CPU bottlenecks include db servers, dynamic web servers, etc. For such applications, CPU configuration and performance should be given priority.

2. Memory

The size of memory is also an important factor affecting Linux performance. If the memory is too small, system processes will be blocked, and applications will become slow or even unresponsive; if the memory is too large, resources will be wasted. The Linux system uses two methods: physical memory and virtual memory. Although virtual memory can alleviate the shortage of physical memory, if it takes up too much virtual memory, the performance of the application will be significantly reduced. To ensure the high-performance operation of the application, physical memory It must be large enough; but too large physical memory will cause a waste of memory resources. For example, on a Linux operating system with a 32-bit processor, any physical memory exceeding 8GB will be wasted. Therefore, to use larger memory, it is recommended to install a 64-bit operating system and enable Linux's large memory kernel support.

Due to the limitation of the processor's addressing range, on a 32-bit Linux operating system, a single application process can only use a maximum of 4GB of memory. In this way, even if the system has larger memory, the application cannot "enjoy" it. Yes, the solution is to use a 64-bit processor and install a 64-bit operating system. Under a 64-bit operating system, the memory usage needs of all applications can be met with almost no restrictions.

Applications that may experience memory performance bottlenecks include NOSQL servers, database servers, cache servers, etc. For such applications, memory size should be given priority.

3. Disk I/O performance

The I/O performance of the disk directly affects the performance of the application. In an application with frequent reading and writing, if the disk I/O performance is not satisfied, the application will stagnate. Fortunately, today's disks use many methods to improve I/O performance, such as common disk RAID technology.
A disk group formed by RAID technology is equivalent to a large hard disk. Users can perform operations such as partition formatting and creating file systems on it. It is exactly the same as a single physical hard disk. The only difference is the I/O performance ratio of the RAID disk group. A single hard drive is much higher, and data security is also greatly improved.

According to different disk combination methods, RAID can be divided into RAID0, RAID1, RAID2, RAID3, RAID4, RAID5, RAID6, RAID7, RAID0 1, RAID10 and other levels. Commonly used RAID levels are RAID0, RAID1, RAID5, RAID0 1. , here is a brief introduction.

“

RAID 0: Improves disk performance and throughput by gluing multiple hard drives into a larger capacity hard drive group. This method is low-cost and requires at least two disks, but it does not have fault tolerance and data recovery functions, so it can only be used in environments that do not require high data security.
RAID 1: That is, disk mirroring. By mirroring the data of one disk to another disk, it maximizes the reliability and repairability of disk data and has high data redundancy. It has spare capacity, but the disk utilization rate is only 50%. Therefore, it has the highest cost and is mostly used in situations where important data is saved.
RAID5: Uses disk segmentation and parity check technology to improve system reliability. RAID5 has high read efficiency and average write efficiency, requiring at least 3 disks. Allows a disk to fail without affecting data availability.
RAID0 1: Combining RAID0 and RAID1 technology becomes RAID0 1, which requires at least 4 hard drives. In addition to the data in this method being distributed on multiple disks, each disk has its own mirror disk, providing full redundancy, allowing one disk failure without affecting data availability, and has fast read/write capabilities.

”

By understanding the performance of each RAID level, you can choose the RAID level that suits you based on the different characteristics of the application, thereby ensuring that the application achieves optimal disk performance.

4． Internet broadband

Various applications under Linux are generally based on the network, so network bandwidth is also an important factor affecting performance. A low-speed, unstable network will lead to blocked access to network applications, while a stable and high-speed network will Bandwidth ensures that applications can run smoothly on the network. Fortunately, today's networks are generally gigabit bandwidth or fiber-optic networks, and the impact of bandwidth issues on application performance is gradually decreasing.

2.2 Operating system related resources

Performance optimization based on the operating system is also multi-faceted and can be measured from several aspects such as system installation, system kernel parameters, network parameters, file system, etc., which are briefly introduced below.

1. System installation optimization

System optimization can start from installing the operating system. When installing a Linux system, disk partitioning and SWAP memory allocation will directly affect the future operating performance of the system. For example, disk allocation can follow the application's needs:

• For applications that require frequent write operations but do not require high data security, the disk can be made into RAID 0;
• For applications with high data security and no special requirements for reading and writing, the disk can be made into RAID 1;
• For applications that have high requirements for read operations but no special requirements for write operations and must ensure data security, you can choose RAID 5;
• For applications that require high reading and writing requirements and high data security requirements, you can choose RAID10/01.

In this way, different RAID levels are set according to different application requirements, and the system is optimized at the bottom of the disk.

With the decrease in memory prices and the increase in memory capacity, the setting of virtual memory SWAP no longer requires the so-called virtual memory to be twice the physical memory. However, the setting of SWAP cannot be ignored. According to experience:

• If the memory is small (physical memory is less than 4GB), generally set the SWAP swap partition size to 2 times the memory;
• If the physical memory is greater than 8GB and less than 16GB, you can set the SWAP size to be equal to or slightly less than the physical memory;
• If the memory size is more than 16GB, in principle, you can set SWAP to 0, but this is not recommended because setting a certain size of SWAP still has a certain effect.

2. Kernel parameter optimization

After the system installation is completed, the optimization work is not over. Next, the system kernel parameters can be optimized. However, the optimization of the kernel parameters must be considered in conjunction with the applications deployed in the system.

For example, if the system deploys an Oracle database application, then you need to configure the system shared memory segment (kernel.shmmax, kernel.shmmni, kernel.shmall), system semaphore (kernel.sem), and file handle (fs. file-max) and other parameters; if you deploy a Web application, you need to optimize the network parameters according to the characteristics of the Web application, such as modifying net.ipv4.ip_local_port_range, net.ipv4.tcp_tw_reuse, net.core.somaxconn, etc. Network kernel parameters.

3. File system optimization

Optimization of the file system is also a focus of system resource optimization. The optional file systems under Linux include ext2, ext3, ReiserFS, ext4, and xfs. Choose different file systems according to different applications.

Linux standard file system starts from VFS, then ext, then ext2. It should be said that ext2 is the standard file system on Linux. ext3 is formed by adding logs on the basis of ext2. From VFS to ext4, the The design ideas have not changed much. They are all based on the design concepts of the early UNIX family based on super blocks and inodes.

The XFS file system is an advanced log file system. XFS provides low-latency, high-bandwidth access to file system data by distributing disk requests, locating data, and maintaining cache consistency. Therefore, XFS is extremely scalable. , very robust, with excellent logging capabilities, strong scalability, and fast write performance.

Currently, server-side ext4 and xfs are the mainstream file systems. How to choose a suitable file system needs to be determined based on the characteristics of the file system and the needs of the business.

2.3, Application software resources

The optimization of the application is actually the core of the entire optimization project. If an application has BUG, ​​then even if all other aspects have reached the optimal state, the performance of the entire application system will still be low. Therefore, the optimization of the application is the performance The top priority of the optimization process puts forward higher requirements for program architecture designers and program developers.

3. Personnel involved in analyzing system performance

3.1, Linux operation and maintenance personnel

In the process of performance optimization, Linux operation and maintenance personnel bear very important tasks.

• First of all, Linux operation and maintenance personnel must understand and master the current operating status of the operating system, such as system load, memory status, process status, CPU load and other information. This information is the basis and basis for detecting and judging system performance;
• Secondly, Linux operation and maintenance personnel also have to master the hardware information of the system, such as disk I/O, CPU model, memory size, network card bandwidth and other parameter information, and then comprehensively evaluate the usage of system resources based on this information;
• Third, as a Linux operation and maintenance personnel, you must also understand the usage of system resources by applications. A more in-depth point is to understand the operating efficiency of applications, such as whether there are program bugs, memory overflows and other problems. By analyzing the system By monitoring resources, you can discover whether there are abnormalities in the application. If there is indeed a problem with the application, the problem needs to be reported to the program developer immediately so that the program can be improved or upgraded.

Performance optimization itself is a complex and tedious process. Only by understanding the system hardware information, network information, operating system configuration information and application information can Linux operation and maintenance personnel carry out targeted optimization of server performance. , which requires Linux operation and maintenance personnel to have sufficient theoretical knowledge, rich practical experience, and the mind to analyze problems carefully.

3.2. System architecture designer

The second type of personnel involved in system performance optimization is the application architect. If Linux operation and maintenance personnel, after comprehensive judgment, find that the performance is affected by the execution efficiency of the application, then the program architecture designer must intervene in a timely manner to gain an in-depth understanding of the program's running status.

• First of all, system architecture designers need to track and understand the execution efficiency of the program. If there is a problem with the execution efficiency, they need to find out where the problem occurred;
• Secondly, if there is really a problem with the architecture design, then the system architecture must be optimized or improved immediately and a better application system architecture designed.

3.3. Software Developer

The last step in system performance optimization involves program developers. After Linux operation and maintenance personnel or architecture designers find program or structural bottlenecks, program developers must immediately intervene to make corresponding program modifications. When modifying a program, the execution efficiency of the program should be used as the benchmark, the logic of the program should be improved, and the code should be optimized in a targeted manner. For example, Linux operation and maintenance personnel found a SQL statement in the system that consumed a lot of system resources. They grabbed the executed SQL statement and found that the execution efficiency of this SQL statement was too poor. This was caused by the low execution efficiency of the code written by the developer. , this requires feedback of this information to the developer. After receiving this question, the developer can perform targeted SQL optimization to optimize the program code.

As can be seen from the above process, the process generally followed for system performance optimization is:

1. First, Linux operation and maintenance personnel check the overall status of the system, mainly making comprehensive judgments from five aspects: system hardware, network equipment, operating system configuration, application architecture and program code. If it is found that there is a problem with system hardware, network equipment or operating system configuration, Linux operation and maintenance personnel can solve the problem independently according to the situation;
2. If it is found to be a program structure problem, it needs to be submitted to the program architecture designer;
3. If it is found that there is a program code execution problem, it will be handed over to the developer for code optimization.
4. This completes a system performance optimization process.

4. Tuning Summary

System performance optimization is a wide-ranging, tedious, and long-term task. Finding the root cause of performance problems is often the most difficult part. Once the cause of the problem is found, the performance problem will be easily solved. Therefore, problem-solving ideas become very important.

For example, for a website system under Linux system, users reported that the website access speed is very slow and sometimes cannot be accessed.

for this problem:

1. The first step is to detect the network. You can use the ping command to check whether the domain name resolution of the website is normal. At the same time, whether the delay of pinging the server address is too large, etc. In this way, first eliminate possible problems in the network; if There is no problem with the network
2. Then enter the second step to check the memory usage of the Linux system. Because the website response speed is slow, it is generally related to memory. Use free, vmstat and other commands to determine whether memory resources are in short supply. If there are no problems with memory resources
3. Enter the third step to check the load status of the system CPU. You can comprehensively determine whether there is an overload problem on the CPU through the output of sar, vmstat, top and other commands. If there is no problem with the CPU
4. Continue to the fourth step to check whether there is a bottleneck in the system's disk I/O. You can check the read and write performance of the disk through iostat, vmstat and other commands. If there is no problem with disk read and write, the performance problem of the Linux system itself is basically eliminated. Finally, you need to What it does is check whether there is a problem with the program itself. Through this kind of thinking, through layer-by-layer detection and step-by-step troubleshooting, performance problems will have "nowhere to hide", and it will become very simple to find the links where performance problems occur.

The above is the detailed content of Linux performance tuning~. For more information, please follow other related articles on the PHP Chinese website!