What are the knowledge points about reading page buffer pool in MySQL?

Buffer pool

We all know that when reading a page, you need to read the page from disk to memory first, and then wait for the CPU to process the data. The process of reading data from disk to memory is very slow, so the pages we read need to be cached, so MySQL has this buffer pool to cache the pages.

First of all, MySQL will apply for a continuous memory space from the operating system when it starts. This space is used as the buffer pool. Put the cached pages into the buffer pool and manage them.

mysql> show variables like 'innodb_buffer_pool_size';
+-------------------------+-----------+
| Variable_name           | Value     |
+-------------------------+-----------+
| innodb_buffer_pool_size | 134217728 |
+-------------------------+-----------+
1 row in set, 1 warning (0.00 sec)

We can see that the default is 134217728 bytes, which is 128MB. If the cache size we apply for is a multiple of 16KB, there will be no fragmentation problem because each page size is 16KB.

Buffer pool composition

Each page contains its corresponding control block information, which is stored in the buffer pool. Each control block manages each page (we use the address to reference each page). The control block is used to store some information about the page. The occupied size of the control block is not included in innodb_buffer_pool_size. MySQL applies for additional space when it starts.

Due to the inability to fully utilize the space, there will be some irregular fragmentation between the control block and the cache page. Because the memory space that MySQL applies to the operating system requires a certain size of control block space, and the specific size cannot be determined, it is inevitable that there will be unusable space.

free linked list

The free linked list, as the name suggests, is a linked list that manages free cache pages. If the cache page is not used, its control block will be connected to the free linked list.

#Connect the control block through a base node to form a free linked list, and store basic information such as the number of free pages.

When we read a page from the disk into the buffer pool, we will take a free control block and fill in the basic information of the corresponding cache page.

Hash processing of cached pages

How does MySQL quickly access a page in the buffer pool and check whether the corresponding page has been cached in the buffer pool?

This is using a hash table, which is a hashmap in Java. The table space page number is processed to form a hash key value, and then the value value is the address of the cache page in the buffer pool.

Management of flush linked lists

I was shocked when I learned this chapter. First of all, it was indeed different from my understanding, and the later MVCC really made me It's an eye-opener. This is a summary I made after studying it once, so it's relatively concise and concise.

When we use SQL statements to modify a certain record, we will modify a certain page or multiple pages. When we modify the page, we will not directly make corresponding modifications to the disk. , because disk IO is too slow, we will first link the modified pages (dirty pages for short), which is similar to the free linked list, that is, a base node connects the control blocks corresponding to the dirty pages together.

This flush linked list represents the linked list that we have not yet updated the page to the disk.

LRU linked list

Because the size of the buffer pool is limited, the size of our cache pages is limited, so we need to separate unused pages Make a elimination. MySQL uses the LRU method for elimination.

LRU is the longest unused elimination strategy. We use a linked list to link cached pages. The most recently accessed pages appear at the front, and the least accessed pages are at the end of the linked list. When the LRU is full, new pages will come in. Eliminate the tail page of the linked list.

We use LRU directly. When MySQL performs pre-reading or full table scanning, a large number of low-frequency pages are read into the LRU linked list, which will cause high-frequency pages to be directly eliminated and replaced by some infrequently used pages. page.

The MySQL optimizer will preload pages that are expected to be accessed by queries into the memory buffer pool to improve query performance. Can be divided into two types:

• Linear read-ahead

When reading pages in an area exceeds the value of the system variable innodb_read_ahead_threshold, the default is 56. That is to say, when we read more than 56 pages in an area, MySQL will asynchronously read all the pages in the next area into memory.

• Random read-ahead

If the buffer pool has cached 13 pages in a certain area, regardless of whether they are sequential or not, as long as there are 13 Once the page is cached, MySQL will be triggered to asynchronously read all pages in this area into MySQL. The system variable innodb_random_read_ahead can be set to turn off random read-ahead. The default is OFF.

So there is an improved partition-based LRU linked list, which divides the linked list into two parts.

One is the young area that is used very frequently, and the other is the old area that is not used very frequently.

正常来说old区占比是37％，所以young区就占63％，我们可以通过innodb_old_blocks_pct来修改，默认就是37。

我们来讲讲这个基于分区的LRU链表。

1. 首先buffer pool初始化，会将读取的页面直接放进old区。

• 但是如果我们对于同一个页面的多条记录进行访问的话，我们就会多次访问同一页多次。但是如果我们是全表扫描的话，是可能会将所有页面缓存进缓存池中的，所以MySQL对于其进行优化。

• 所以MySQL对于当页面第一次读入old区并在一定时间间隔（innodb_old_blocks_pct）内的多次访问来说是不会将其放入young区进行缓存的。innodb_old_blocks_pct的值默认为1000，就是刚来的来一秒内的多次访问是不会将其转移到young区的。

1. 如果多次访问就会将old区的页升级到young区。当young区的页面被访问，只有young链表后1/4的页面被访问时才会将其转置到young区链表头，不然就不会改动，减少一些调整链表的性能损失。

刷新脏页

MySQL会启动后台线程进行脏页，也就是修改的页面进行刷新到磁盘。

以下有两种方式刷新脏页：

• 从LRU的尾部扫描一些页面，刷新其中的脏页到磁盘中。

• 在LRU链表的old区域尾部，即不经常使用的页面中，后台线程会查找是否存在脏页，如果有，则将其更新至磁盘。控制扫描区域尾部数量的方法是更改系统变量innodb_lru_scan_depth。

• 从flush链表中更新到磁盘。

• 我们上面说了flush连接这脏页的控制块，我们就可以将连接这flush链表的脏页进行更新。

疑问：为什么要两种方式更新呢？我刚开始不懂这是我回过头来看的时候就懂了

首先我们脏页是缓存在buffer pool中的，但是我们buffer pool空间是有限的，又因为我们使用的是LRU的方式，又因为从flush链表将脏页同步到磁盘效率实在不高，所以不会很经常去更新脏页。如果我们不更新直接将其从LRU的链表抛弃也就是从缓存池中直接扔了，但是它是脏页就无法同步到磁盘了，同时flush链表链接的也会出现问题。

所以在LRU淘汰很久未使用的页有个前提就是它不是一个脏页。为了淘汰这些页面，我们需要检查LRU链表的末尾是否存在脏页并进行更新。

flush链表更新那就是它的本职工作了，它存这个也是干这个的，应该没有什么问题。

当系统十分繁忙，buffer pool使用量不足的时候，因为磁盘IO太慢了，所以会出现一种情况，就是大量的用户线程也在进行这个同步脏页的活。如果未进行脏页同步并淘汰缓冲池的页面，则无法读取该页面。

多个buffer pool实例

我们可以设置多个buffer pool来实现多实例提高性能。

mysql> show variables like 'innodb_buffer_pool_instances';
+------------------------------+-------+
| Variable_name                | Value |
+------------------------------+-------+
| innodb_buffer_pool_instances | 1     |
+------------------------------+-------+
1 row in set, 1 warning (0.00 sec)

我们可以设置innodb_buffer_pool_instances系统变量来控制实例变量。

但是当buffer pool的大小小于1G的时候，设置2个实例也是没有用的（会被恢复成1个），多实例的情况是建立在大内存的情况下的。

动态调整buffer pool大小

在MySQL5.7.5后，MySQL中的buffer pool的大小是以chunk来分配了，如下图。

一个buffer pool是由多个chunk组成的，所以MySQL向操作系统申请连续的内存空间，就是以chunk的方式来申请的，这样我们可以在MySQL运行时调整buffer pool的大小。在运行时更改chunk大小不可行，并且会造成性能浪费。？

innodb_buffer_pool_size / innodb_buffer_pool_instances = 每个实例buffer pool的大小。

每个实例的大小 / innodb_buffer_pool_chunk_size = 每个实例由多少个chunk构成。

不是弄很明白，怎么动态调整大小，我调整了但是mysqld占用内存大小还是只能重启才能生效，我不会。

查看buffer pool具体的信息

show engine innodb status;

The above is the detailed content of What are the knowledge points about reading page buffer pool in MySQL?. For more information, please follow other related articles on the PHP Chinese website!