[Linux System Optimization] Liberate your memory space—swap and buffer optimization guide

Have you ever encountered a Linux system that runs slowly or has insufficient memory? It may be because the swap and buffer in the system are not well configured. This article will give you an in-depth understanding of swap and buffer, and how to optimize them to take your system performance to the next level.

What is the memory mechanism of Linux?

We know that reading and writing data directly from physical memory is much faster than reading and writing data from the hard disk. Therefore, we hope that all data reading and writing are completed in memory, and memory is limited, so Introduced the concepts of physical memory and virtual memory.

Physical memory is the memory size provided by the system hardware. It is real memory. Compared with physical memory, there is a concept of virtual memory under Linux. Virtual memory is a strategy proposed to meet the shortage of physical memory. It is A piece of logical memory is virtualized using disk space. The disk space used as virtual memory is called swap space.

As an extension of physical memory, Linux will use the virtual memory of the swap partition when the physical memory is insufficient. More specifically, the kernel will write the temporarily unused memory block information to the swap space. In this way, the physical memory will be Once released, this memory can be used for other purposes. When the original content is needed, the information will be read from the swap space into physical memory again.

Linux's memory management adopts a paging access mechanism. In order to ensure that physical memory can be fully utilized, the kernel will automatically swap infrequently used data blocks in physical memory into virtual memory at the appropriate time, and will Frequently used information is retained to physical memory.

To have an in-depth understanding of the Linux memory operating mechanism, you need to know the following aspects:

The Linux system will perform page swap operations from time to time to maintain as much free physical memory as possible. Even if there is nothing that requires memory, Linux will swap out temporarily unused memory pages. This avoids the time required to wait for the exchange.

Linux page swapping is conditional. Not all pages are swapped to virtual memory when not in use. The Linux kernel only swaps some infrequently used page files to virtual memory based on the "most recently used" algorithm. Sometimes We will see this phenomenon: Linux still has a lot of physical memory, but a lot of swap space is also used. In fact, this is not surprising. For example, when a process that takes up a lot of memory needs to consume a lot of memory resources when running, some uncommon page files will be swapped into virtual memory, but later this process that takes up a lot of memory resources will be swapped into virtual memory. When the process ends and a lot of memory is released, the page file that was just swapped out will not be automatically swapped into the physical memory. Unless this is necessary, the system's physical memory will be much free at this moment, and the swap space is also being used. The phenomenon just mentioned occurred. There's nothing to worry about at this point, as long as you know what's going on.

The pages in the swap space will first be swapped to physical memory when used. If there is not enough physical memory to accommodate these pages at this time, they will be swapped out immediately. As a result, there may not be enough space in the virtual memory to Storing these swap pages will eventually cause problems such as false crashes and service abnormalities in Linux. Although Linux can recover by itself within a period of time, the recovered system is basically unusable.

Therefore, it is very important to properly plan and design the use of Linux memory.

In the Linux operating system, when an application needs to read data in a file, the operating system first allocates some memory, reads the data from the disk into these memories, and then distributes the data to the application; when needed When writing data to a file, the operating system first allocates memory to receive user data, and then writes the data from memory to disk. However, if there is a large amount of data that needs to be read from the disk to the memory or written from the memory to the disk, the read and write performance of the system becomes very low, because whether it is reading data from the disk or writing data to the disk, it is a very long process. A process that consumes time and resources. In this case, Linux introduced the buffers and cached mechanisms.

Buffers and cached are both memory operations, used to save files and file attribute information that have been opened by the system. In this way, when the operating system needs to read certain files, it will first search in the buffers and cached memory areas. If found, Directly read out and send it to the application. If the required data is not found, it will be read from the disk. This is the caching mechanism of the operating system. Through caching, the performance of the operating system is greatly improved. But the contents of buffers and cached buffers are different.

Buffers is used to buffer block devices. It only records the metadata of the file system and tracking in-flight pages, while cached is used to buffer files. To put it more simply: buffers are mainly used to store the contents of the directory, file attributes and permissions, etc. And cached is directly used to remember the files and programs we have opened.

In order to verify whether our conclusion is correct, we can open a very large file through vi to see the changes in cache, and then vi the file again to feel the similarities and differences in the speed of opening it twice, and whether it is the second time it is opened. The speed is significantly faster than the first time? Then execute the following command:

find / -name .conf to see if the value of buffers changes, and then execute the find command repeatedly to see the difference in display speed between the two times.

When did Linux start using virtual memory (swap)?

[root@wenwen ~]# cat /proc/sys/vm/swappiness  
60

The 60 above means that swap will be used when 40% of the physical memory is used (refer to network information: when the remaining physical memory is less than 40% (40=100-60), the swap space will be used) swappiness=0 When swappiness = 100, it means actively using the swap partition and moving the data on the memory to the swap space in a timely manner.

The larger the value, the more likely it is to use swap. It can be set to 0, which does not prohibit the use of swap, but only minimizes the possibility of using swap.

Normally: the swap partition setting is recommended to be twice the memory (when the memory is less than or equal to 4G). If the memory is greater than 4G, the swap only needs to be larger than the memory. In addition, try to lower swappiness as much as possible, so that the system performance will be better.

B. Modify swappiness parameter

#临时性修改：  
[root@wenwen ~]# sysctl vm.swappiness=10  
vm.swappiness = 10  
[root@wenwen ~]# cat /proc/sys/vm/swappiness  
10  
#永久性修改：  
[root@wenwen ~]# vim /etc/sysctl.conf  
加入参数：  
vm.swappiness = 35 
然后在直接：  
[root@wenwen ~]# sysctl -p /etc/sysctl.conf  
#查看是否生效：  
cat /proc/sys/vm/swappiness  
35 

It takes effect immediately and can also take effect after restarting.

How to release memory?

General systems do not automatically release the key configuration file /proc/sys/vm/drop_caches of memory. This file records the parameters of cache release. The default value is 0, which means the cache is not released. Its value can be any number between 0 and 3, representing different meanings:

• 0 – Do not release
• 1 – Release page cache
• 2 – Release dentries and inodes
• 3 – Release all cache

Practical operation:

Obviously there is a lot of free memory

How to release swap?

Premise: First of all, make sure that the remaining memory is greater than or equal to the swap usage, otherwise it will crash! According to the memory mechanism, once the swap partition is released, all files stored in the swap partition will be transferred to physical memory. Releasing swap is usually accomplished by remounting the swap partition.

a. Check where the current swap partition is mounted? b. Shut down this partition c. Check the status: d. Check whether the swap partition is shut down. The bottom line shows all e. Mount swap to /dev/sda5 f. Check whether the mounting is successful

Through the introduction of this article, you have already understood the basic concepts and uses of swap and buffer in Linux systems and how to check their usage. At the same time, we have shared some practical optimization suggestions to help you further improve system performance and memory utilization. Hope this article helps you!

The above is the detailed content of [Linux System Optimization] Liberate your memory space—swap and buffer optimization guide. For more information, please follow other related articles on the PHP Chinese website!