How to solve the problem of high Linux Swap space utilization

What is swap?

swap space is an area on the disk, which can be a partition, a file, or a combination of them .

To put it simply, when the system's physical memory is tight, Linux will save infrequently accessed data in the memory to swap, so that the system has more physical memory to serve each process, and when the system needs When accessing the content stored on the swap, the data on the swap is loaded into the memory. This is what we often call swap out and swap in.

Why is swap needed?

To answer this question, we need to answer what benefits swap brings to us.

For some large applications (such as LibreOffice, video editor, etc.), a large amount of memory will be used during the startup process, but this memory is often only used during startup, and will not be used during subsequent operations. This memory is rarely used again. With swap, the system can save this part of the memory data that is not used in this way to the swap, thereby releasing more physical memory for the system to use.

The hibernation function of many distributions (such as ubuntu) relies on the swap partition. When the system hibernates, the data in the memory will be saved to the swap partition, and then the data will be saved when the system starts next time. Loading into memory can speed up the system startup, so if you want to use the hibernation function, you must configure a swap partition, and the size must be greater than or equal to physical memory.

In some cases, the physical memory is limited, but what should I do if I want to run a memory-consuming program? At this time, you can achieve the goal by configuring enough swap space. Although it is a little slower, it can at least run.

Although physical memory is sufficient in most cases, there are always some unexpected situations, such as a process requiring more memory than expected, or a process having a memory leak, etc. When the memory When it is not enough, the kernel's OOM killer will be triggered. According to the configuration of the OOM killer, some processes will be killed or the system will be restarted directly (the default is to kill the process that consumes the most memory first). However, with swap, you can Using swap as memory, although it is a bit slower, at least gives us an opportunity to debug, kill the process or save the current work progress.

If you have seen Linux memory management, you will know that the system will use as much free memory as possible for cache to speed up the I/O speed of the system, so if you can move less commonly used memory data to On swap, more physical memory will be used for cache, thereby improving the overall performance of the system.

Disadvantages of swap?

The advantages of swap have been introduced above, but what about the disadvantages of swap? Swap is stored on the disk. The speed of the disk is several orders of magnitude slower than that of the memory. If you continue to read and write swap, it will definitely have an impact on the performance of the system, especially when the system memory is very tight. The frequency of swap space will be very high, causing the system to run very slowly, as if dead. At this time, adding physical memory is the only solution.

Since the system will automatically move infrequently used memory data to the swap, for desktop programs, it may cause a small pause when you minimize a program and then open it again, because you need to move the memory data on the swap. The data is reloaded into memory.

Should we swap?

The above introduces what swap is and its advantages and disadvantages, so should we configure swap? The answer is: it depends.

The following discusses the choice of swap for the server and desktop environment in three situations: insufficient memory, barely enough memory, and sufficient memory.

Not enough memory

Whether it is a desktop or a server, when the physical memory is obviously not enough and you want to run the program, adding swap is the only option. Slower is better than not working at all.

The memory is barely enough

It is recommended to configure swap, so that the kernel will move infrequently used data from the memory to the swap, so that more physical memory is available for system calls and improve system performance. At the same time, it also avoids abnormal process exit due to occasional insufficient physical memory and improves system stability. However, for servers, it is necessary to limit or monitor the usage of swap space. When the swap space usage exceeds expectations or swap in/out is frequent, , measures must be taken in time, otherwise it will have a great impact on performance

Abundant memory

Theoretically, if there is enough physical memory and no hibernation function is needed, then swap is useless, but the key issue is It is difficult for us to ensure that the physical memory is sufficient under any circumstances, because there are always unexpected situations, such as some processes consuming more memory than expected, server pressure exceeding expected, memory leaks, etc.

Currently, we are obviously running out of memory. What causes the running out of memory? Why does mysql directly cause the server to run out of memory?

So why does swap occur in our mysql server?

Suppose our physical memory is 16G and swap is 4G. If MySQL itself already occupies 12G of physical memory, and at the same time other programs or system modules require 6G of memory, then the operating system may map part of the address space owned by MySQL to swap.

To put it bluntly, the system thinks that the space occupied by mysql is too large and does not allow you to do special things. It must make room for other necessary process areas of mine to use memory, so you go to a slower swap to play. Bar!

The largest memory occupier in mysql is innodb_buffer_pool_size, so you should consider whether this value is set unreasonably at the first time?

MySQL’s memory consumption is divided into:

• #1. Session-level memory consumption: such as sort_buffer_size, etc., each session will open a sort_buffer_size To perform sorting operations

• 2. Global memory consumption: for example: innodb_buffer_pool_size, etc., global shared memory segment

This is what I think we The unprofessional part of the DBA was that he did not consider the first situation and check the memory consumption at the session level, but directly told me to reduce the innodb_buffer_pool_size

What does InnoDB's buffer pool cache? What is the use? What is the appropriate setting?

Cache table data and index data, load the data on the disk into the buffer pool, avoid disk IO for each access, and accelerate access.

The concurrency performance of MySQL is directly proportional to the memory size allocated by the Buffer Pool. The larger the memory allocated, the better the concurrency performance. Should all 99% of the machine's memory be allocated to the Buffer Pool?

of course not! Not to mention that the operating system kernel also requires several gigabytes of memory. In addition to Buffer Pool, MySQL also has many other memory data structures. These all require memory, so the above idea is definitely not feasible!

A more reasonable ratio should be that the memory size of the Buffer Pool accounts for 50% ~ 60% of the total memory of the machine.

You can check the hit status through show engine innodb status\G;. When the hit does not reach more than 97%, you can consider adding memory. Of course, this is also related to the business. For example, the write volume to a master is large, and the read Less is a special case.

In other cases, if it does not reach more than 97%, and in the case of large reads, if it does not reach more than 98%, it means that the buffer is not enough. It can be expanded. On the other hand, if If you divide the memory by 20%, the hit rate can reach 100%, and there are a large number of free pages, which means that it is enough. In addition, you can also calculate according to the free pages to reduce the memory. Use that memory Why don’t you use it?

The above is the detailed content of How to solve the problem of high Linux Swap space utilization. For more information, please follow other related articles on the PHP Chinese website!