转贴一个有关MYSQL的文章.E文的.MySQLs Query Cache

落杰大大_1364

落杰大大_1364

2016-06-21

1239人浏览

原创

cache|mysql

A typical scenario
Boss: Our new website is crawling! How can it be, we have four state-of-the-art web servers - what's the problem?
You: Well, the web servers are fine - it's the database server that's struggling.
Boss: What? You told me this MySQL thing was fast, that we didn't need Oracle, and now you say it can't cope! How can this be?
You: Well, the web servers are behaving so well that they're pushing through lots of queries, and the database can't manage to process all of them at the same time. It's only one database, and lots of web servers...
Boss: It's too late to buy Oracle now - what are we going to do!?
Big Boss to Boss(in the boss's mind): This project has been a disaster from the beginning - now you want me to delay it while we install a new database, and spend a whole lot more! Do you think we're made of money!? I'm calling in someone who knows what they're doing - you're history buddy.
Colleague (about to take your job): Wait, I think I can solve the problem!

So, what does your colleague know that you don't? How can he save the day and let the boss get all the credit? Our scenario is too imprecise to generalize, and there are many possible solutions. You can read about optimizing queries and indexes, optimizing by improving the hardware, and tweaking the MySQL variables, using the slow query log, and of course, there are other methods such as replication. However, MySQL 4 provides one feature that can prove very handy - a query cache. In a situation where the database has to repeatedly run the same queries on the same data set, returning the same results each time, MySQL can cache the result set, avoiding the overhead of running through the data over and over. Usually, you would want to implement some sort of caching on the web server, but there are times when this is not possible, and then it is the query cache you will look to for help.

Setting up the query cache
To make sure MySQL uses the query cache, there are a few variables you need to set in the configuration file (usually my.cnf or my.ini). First, is the query_cache_type. There are three possible settings: 0 (for off, do not use), 1 (for on, cache queries) and 2 (on demand, discussed more below). To ensure it is always on, place:

query-cache-type = 1
in the configuration file. If you started the server having only made this change, you would see the following cache variables set:

mysql> SHOW VARIABLES LIKE '%query_cache%';
+-------------------+---------+
| Variable_name     | Value   |
+-------------------+---------+
| have_query_cache  | YES     |
| query_cache_limit | 1048576 |
| query_cache_size  | 0       |
| query_cache_type  | ON      |
+-------------------+---------+
4 rows in set (0.06 sec)

Note that these are results from MySQL 4.0.x - you'll see more in versions 4.1.x and beyond. The query_cache_type will be set to ON or OFF as appropriate. However, there is one more to set, and that is the query_cache_size. If set to 0 (the default), the cache will be disabled. This variable determines the memory, in bytes, used for the query cache. For our purposes, we will set it to 20 MB:

query-cache-size = 20M
The amount is shown in bytes:

mysql> SHOW VARIABLES LIKE '%query_cache%';
+-------------------+----------+
| Variable_name     | Value    |
+-------------------+----------+
| have_query_cache  | YES      |
| query_cache_limit | 1048576  |
| query_cache_size  | 20971520 |
| query_cache_type  | ON       |
+-------------------+----------+
4 rows in set (0.06 sec)

The Query cache in action (almost)
For this tutorial, I used a dump from Wikipedia, the open content encyclopedia (you can find the dumps here. I am using a fairly slow machine, with nothing else happening on it, to minimize interference in the results. Let's run the same query twice, and see how much improvement we see the second time:

SELECT * FROM cur;
...
14144 rows in set (2.96 sec)
Now we run the same query again:
SELECT * FROM cur; 14144 rows in set (3.02 sec)

Now we run the same query again:

SELECT * FROM cur; 14144 rows in set (3.02 sec)

What is happening? We would expect the second query to take noticeably less time. Let's examine some of the status variables to get a better picture.

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 0        |
| Qcache_inserts          | 2        |
| Qcache_hits             | 0        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 2        |
| Qcache_free_memory      | 20962720 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 1        |
+-------------------------+----------+
8 rows in set (0.00 sec)

The two queries we ran are both recorded (by Qcache_inserts), but neither of them have been cached. (You may get different results if other queries have been running.) The problem is that the result set is too big. I used the Wikipedia Esperanto dump (4MB compressed - the English dump is 135MB, and even though my English is better than my Esperanto, bandwidth is expensive in South Africa!), but it is immaterial, as even that is more than the query cache can handle by default. There are two limits in play here - the limit for each individual query is determined by the value of query_cache_limit, which is 1MB by default. Moreover, the limit of the cache in total is determined by query_cache_size, which we have seen already. The former limit applies here. If a result set is greater than 1M, it is not cached.

The Query cache in action (really)
Let's try a smaller query:

SELECT cur_is_new FROM cur WHERE cur_user_text > 'Y'
...
2336 rows in set (0.38 sec)

Let's see if this one was cached:

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 1        |
| Qcache_inserts          | 3        |
| Qcache_hits             | 0        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 2        |
| Qcache_free_memory      | 20947592 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 4        |
+-------------------------+----------+
8 rows in set (0.00 sec)

There is now a query in the cache. If it took 0.38 seconds to run the first time, let's see if we notice an improvement the second time:

SELECT cur_is_new FROM cur WHERE cur_user_text > 'Y'
...
2336 rows in set (0.11 sec)

Much better! And, looking at the status again:

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 1        |
| Qcache_inserts          | 3        |
| Qcache_hits             | 1        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 2        |
| Qcache_free_memory      | 20947592 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 4        |
+-------------------------+----------+
8 rows in set (0.06 sec)

The cache has been hit once. The status variables above should be fairly self-explanatory. Available memory for the cache has gone from 20962720 to 20947592 bytes. The most useful variable for future tuning is Qcache_lowmem_prunes. Each time a cached query is removed from the query cache, (because MySQL needs to make space for another), this value will be incremented. If it increases quickly, and you still have memory to spare, you can up the query_cache_size, while if it never increases, you can reduce the cache size.

Let's run the query again, with a slight difference, as follows:

SELECT cur_is_new from cur where cur_user_text > 'Y'
...
2336 rows in set (0.33 sec)

That took longer than we would have expected. Let's look at the status variables to see what's up:

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 2        |
| Qcache_inserts          | 4        |
| Qcache_hits             | 1        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 2        |
| Qcache_free_memory      | 20932976 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 6        |
+-------------------------+----------+

The query has not made use of the cache - in fact, MySQL has inserted another query in the cache! The problem here is that MySQL's query cache is case-sensitive (in fact it is byte sensitive). The query must be identical in every way - no extra spaces, no changes in case. Therefore, the above query is treated as a different query. This fact alone should be enough for you to adopt a certain convention, and ensure all application developers make use of it. I use caps for MySQL keywords, and lower case for table and field names.

Clearing the Query cache
The cache cannot stay in memory indefinitely. Luckily, MySQL is clever enough to clear it when you make any changes to the tables used in a cache query. If we insert a new record to the cur table, MySQL will clear the affected queries (and only the affected queries) from the cache:

mysql> INSERT INTO cur(cur_user_text)
    VALUES ('xxx');
Query OK, 1 row affected (0.06 sec)

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 0        |
| Qcache_inserts          | 4        |
| Qcache_hits             | 1        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 2        |
| Qcache_free_memory      | 20962720 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 1        |
+-------------------------+----------+

Any of INSERT, UPDATE, DELETE, TRUNCATE, ALTER, DROP TABLE or DROP DATABASE potentially remove queries from the cache. You can manually clear the query cache with RESET QUERY CACHE.

Query Cache on demand
Earlier we saw there were three values for the query_cache_type. On, off and on demand. The latter option means that queries will only be cached if SQL_CACHE is specified in the query. Let's restart the server, with

query-cache-type = 2
in the configuration. Restarting the server flushes all the status variables. We run our previous query again:

SELECT cur_is_new FROM cur WHERE cur_user_text > 'Y'
...
2336 rows in set (0.27 sec)

It is back to a longer time again, as the cache has been flushed.

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 0        |
| Qcache_inserts          | 0        |
| Qcache_hits             | 0        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 1        |
| Qcache_free_memory      | 20962720 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 1        |
+-------------------------+----------+

Nothing has been recorded. To store a query in the cache, we need to run the query with SQL_CACHE, as follows:

SELECT SQL_CACHE cur_is_new FROM cur WHERE cur_user_text > 'Y'
...
2336 rows in set (0.33 sec)

This time it has been stored in the cache.

mysql> SHOW STATUS LIKE '%qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_queries_in_cache | 1        |
| Qcache_inserts          | 1        |
| Qcache_hits             | 0        |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 1        |
| Qcache_free_memory      | 20947592 |
| Qcache_free_blocks      | 1        |
| Qcache_total_blocks     | 4        |
+-------------------------+----------+

If the cache type is set to 1, there may be times when you know the query you are running will not be repeated, or more infrequently. In these situations, you can ask MySQL not to store the results in the cache, even if they adhere to the size limitations, with the SQL_NO_CACHE clause in the SELECT statement.

Block allocation and the Query cache
MySQL allocates results into the cache in blocks, during retrieval. This allocation comes at an overhead (see the quicker time to run the above query when it was not being cached). You should not enable the query cache unless you can make good use of it. The number of free blocks (Qcache_free_blocks) can be an indication of fragmentation - a high number in relation to the total number of the blocks means that space is being wasted. In MySQL 4.1, there is another cache-related variable: query_cache_min_res_unit. This allows you to set a minimum block size. The default is 4KB. If most of your query results are small, and you see fragmentation, you should decrease this. The converse applies if most of your result sets are large. To defragment a query cache, you can use FLUSH QUERY CACHE (FLUSH TABLES has the same effect on the cache).

There are situations when a query cannot be cached, all of which make perfect sense, such as when returning the current time, a random number, user variables, or when dumping to a file. Any queries making use of the following functions, or of the following types, will not be cached:

User-Defined Functions
BENCHMARK
CONNECTION_ID
CURDATE
CURRENT_DATE
CURRENT_TIME
CURRENT_TIMESTAMP
CURTIME
DATABASE
ENCRYPT (with one parameter)
FOUND_ROWS
GET_LOCK
LAST_INSERT_ID
LOAD_FILE
MASTER_POS_WAIT
NOW
RAND
RELEASE_LOCK
SYSDATE
UNIX_TIMESTAMP (without parameters)
USER
query contains user variables
query references the mysql system database
query of the form SELECT ... IN SHARE MODE
query of the form SELECT ... INTO OUTFILE ...
query of the form SELECT ... INTO DUMPFILE ...
query of the form SELECT * FROM AUTOINCREMENT_FIELD IS NULL
queries inside transactions (in MySQL 4.0.x)
Wisely used, the query cache can make a substantial difference to struggling applications. Good luck!



相关文章

PHP速学视频免费教程(入门到精通)
PHP速学视频免费教程(入门到精通)

PHP怎么学习?PHP怎么入门?PHP在哪学?PHP怎么学才快?不用担心,这里为大家提供了PHP速学教程(入门到精通),有需要的小伙伴保存下载就能学习啦!

下载

相关标签:

本站声明:本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系admin@php.cn

相关专题

更多
Go-Zero API开发使用教程
Go-Zero API开发使用教程

本专题聚焦Go-Zero的API开发,拆解.api描述文件的语法结构、路由与分组定义、参数绑定与校验规则,说明统一响应体、JWT鉴权、跨域与接口文档生成的落地方式,并给出网关聚合与接口测试的实践建议。

2026.11.12

140

15

Go-Zero安装与goctl环境配置教程
Go-Zero安装与goctl环境配置教程

本专题围绕Go-Zero的安装与工具链配置展开,讲清Go版本要求、goctl安装方式、代理镜像、protoc依赖与环境变量设置,说明多版本共存、升级回退与CI流水线安装的做法,并整理安装阶段的常见报错与验证步骤。

2026.11.12

100

15

Go-Zero零基础入门教程
Go-Zero零基础入门教程

本专题整理Go-Zero的入门知识,从框架定位、环境准备到跑通第一个API服务,讲清目录分层、配置加载、日志与中间件的基本用法,并对比Gin、Kratos的差异,帮助Go开发者建立整体认知,判断它是否适合当前项目。

2026.11.12

80

15

Postman身份认证怎么配置
Postman身份认证怎么配置

本专题从实际API接口调试场景出发,讲解Postman身份认证的完整配置流程,包括常见认证协议介绍、授权参数设置、访问令牌填写、OAuth 2.0授权及认证失败排查,帮助用户正确完成接口授权与访问验证。

2026.11.06

160

12

Postman请求头Headers怎么设置
Postman请求头Headers怎么设置

围绕Postman HTTP请求头配置展开,讲解Headers选项的具体使用方式,包括Key和Value参数填写、请求头信息添加、常见Header字段设置及请求头修改方法,帮助用户解决接口调试过程中遇到的请求头配置问题。

2026.11.06

140

13

Postman接口请求发送教程
Postman接口请求发送教程

详细介绍Postman接口请求的创建与发送方法,涵盖GET和POST请求、URL配置、请求参数、Headers请求头、Body请求体设置以及响应结果查看,帮助用户掌握Postman接口请求的完整操作流程。

2026.11.06

120

10

DBeaver怎么导入数据
DBeaver怎么导入数据

DBeaver数据导入专题,整理Excel、CSV、SQL、TXT等常见文件的数据导入方法,涵盖目标数据库选择、数据表创建、字段映射、数据类型设置及导入参数配置等操作,帮助用户将本地文件中的数据写入数据库,并了解批量导入、重复数据处理和导入失败等常见问题的解决方法。

2026.11.05

120

12

DBeaver安全防护教程大全
DBeaver安全防护教程大全

DBeaver安全防护专题,围绕数据库连接安全、账号密码保护和访问权限管理展开,整理SSL/TLS加密连接、SSH隧道、数据库凭据保存及证书验证等配置方法,同时介绍敏感信息保护、远程连接风险和高权限账号使用等注意事项,帮助用户提高数据库日常管理的安全性。

2026.11.05

100

12

DBeaver连接创建与配置教程
DBeaver连接创建与配置教程

本专题聚焦DBeaver数据库连接配置与管理,覆盖从新建连接、选择数据库类型到填写连接参数和验证连接的完整流程。针对MySQL、PostgreSQL、SQL Server等常见数据库,整理相应的连接设置方法,并提供驱动缺失、认证失败和连接异常等问题的排查方向。

2026.11.04

120

16

热门下载

更多
网站特效
/
网站源码
/
网站素材
/
前端模板

精品课程

更多
相关推荐
/
热门推荐
/
最新课程