bitsCN.com
转载请署名:印风
-----------------------------------------------------------
最近碰到一个问题,线上一台机器在等待信号量时间过长,mysql的监控线程认为此时mysqld已经hang住了,于是自杀重启。这里涉及到一个有趣的问题,也就是mysql如何对读写锁进行处理。
主要包括三个部分:
1. 建锁
2. 加锁
3. 解锁
4. 监控锁
以下内容基于Percona5.5.18进行分析
1.创建锁
锁的创建实际上就是初始化一个RW结构体(rw_lock_t),实际调用函数如下:
# define rw_lock_create(K, L, level) /
rw_lock_create_func((L),#L)
在rw_lock_create上有三个参数,在实际场景锁时只用到第2个参数
其中K表示mysql_pfs_key_t,level显示当前的操作类型(起码看起来是的,在文件sync0sync.h中定义),看起来k是为performance schema准备的,而k代表了当前操作所在的层次。
例如:purge线程的读写锁创建:
rw_lock_create(trx_purge_latch_key,
&purge_sys->latch,SYNC_PURGE_LATCH);
我们进去rw_lock_create_func看看到底是怎么创建的。
可以看到这个函数的逻辑其实很简单:
lock->lock_word =X_LOCK_DECR; //关键字段
用于限制读写锁的最大并发数,代码里的注释如下:
/* We decrement lock_word by this amountfor each x_lock. It is also the
start value for the lock_word, meaning thatit limits the maximum number
of concurrent read locks before the rw_lockbreaks. The current value of
0x00100000 allows 1,048,575 concurrentreaders and 2047 recursive writers.*/
在尝试加锁时会调用rw_lock_lock_word_decr减少lock_word
在初始化一系列变量后,执行:
lock->event = os_event_create(NULL);
lock->wait_ex_event = os_event_create(NULL);
os_event_create用于创建一个系统信号,实际上最终创建的还是互斥量(os_fast_mutex_init(&(event->os_mutex));以及条件变量(os_cond_init(&(event->cond_var));)
最后将lock加入到全局链表rw_lock_list中
2.加锁
加锁函数由宏定义,实际调用函数为:
1)写锁
# define rw_lock_x_lock(M) /
rw_lock_x_lock_func((M),0, __FILE__, __LINE__)
当申请写锁时,执行如下步骤:
(1).调用rw_lock_x_lock_low函数去获取锁,如果得到锁,则rw_x_spin_round_count += i后直接返回,如果得不到锁,继续执行
(2).loop过程中只执行一次rw_x_spin_wait_count++
(3).在毫秒级别的loop多次等待
while (i && lock->lock_word if(srv_spin_wait_delay) {
ut_delay(ut_rnd_interval(0,
srv_spin_wait_delay));
}
i++;
}
这里涉及到两个系统变量:
innodb_sync_spin_loops(SYNC_SPIN_ROUNDS)
innodb_spin_wait_delay(srv_spin_wait_delay)
在SYNC_SPIN_ROUNDS循环里调用函数ut_delay,这个函数很简单,就是做了delay*50次空循环
Ut_delay(uint delay):
for(i = 0; i j+= i;
UT_RELAX_CPU();
}
其中,UT_RELAX_CPU()会调用汇编指令来独占CPU,以防止线程切换
(4).如果loop的次数等于SYNC_SPIN_ROUNDS,调用os_thread_yield(实际调用pthread_yield,导致调用线程放弃CPU的占用)将线程挂起;否则挑到1继续loop
(5).在sync_primary_wait_array里获取一个cell(占个坑?)。调用sync_array_reserve_cell,看起来有1000个坑位(sync_primary_wait_array->n_cells)
(6).再次调用rw_lock_x_lock_low函数尝试获取锁,若成功获得,则返回
(7).调用sync_array_wait_event等待条件变量,然后返回1继续loop
具体的加锁函数(rw_lock_x_lock_low)稍后分析
2)读锁
# define rw_lock_s_lock(M) /
rw_lock_s_lock_func((M),0, __FILE__, __LINE__)
这个函数定义在sync0rw.ic里,函数也很简单,如下:
if (rw_lock_s_lock_low(lock, pass, file_name, line)) {
return; /* Success */
}else {
/* Did not succeed, try spin wait */
rw_lock_s_lock_spin(lock, pass, file_name, line);
return;
}
这里首先调用rw_lock_s_lock_low进行加锁,如果加锁不成功,则调用rw_lock_s_lock_spin进行等待,rw_lock_s_lock_spin的代码逻辑与rw_lock_x_lock_func有些相似,这里不再赘述。
在rw_lock_s_lock_spin里会递归的调用到rw_lock_s_lock_low函数;
看起来实际的加锁和解锁操作是通过对计数器来控制的,
(1)在函数rw_lock_s_lock_low中
rw_lock_lock_word_decr (lock, 1),对lock->lock_word减去1
减数成功返回true,否则返回false
这部分的逻辑还是很简单的。
(2)在函数rw_lock_x_lock_low中,调用:
rw_lock_lock_word_decr(lock, X_LOCK_DECR),对lock->lock_word减去X_LOCK_DECR
减数成功后,执行:
rw_lock_set_writer_id_and_recursion_flag(lock,pass ? FALSE : TRUE)来设置:
lock->writer_thread = s_thread_get_curr_id()
lock->recursive = TRUE
然后调用rw_lock_x_lock_wait函数等待lock->lock_word=0,也就是说等待所有的读锁退出。
看到一个比较有意思的现象,在.ic的代码里看到使用了宏
INNODB_RW_LOCKS_USE_ATOMICS,这是跟gcc的版本相关的,通过使用gcc的内建函数来实现原子操作。
3.解锁
解锁操作包括解除读锁(#define rw_lock_s_unlock(L) rw_lock_s_unlock_gen(L, 0))和解除写锁操作(#definerw_lock_x_unlock(L) rw_lock_x_unlock_gen(L, 0))
实际调用函数为rw_lock_s_unlock_func和rw_lock_x_unlock_func
1)解除读锁(rw_lock_s_unlock_func)
增加计数rw_lock_lock_word_incr(lock, 1)
2)解除写锁(rw_lock_x_unlock_func)
执行如下操作
(1)如果是最后一个递归调用锁的线程,设置lock->recursive= FALSE; 代码里的注释如下:
/* lock->recursive flag also indicatesif lock->writer_thread is
valid or stale. If we are the last of the recursive callers
then we must unset lock->recursive flag to indicate that the
lock->writer_thread is now stale.
Note that since we still hold the x-lock we can safely read the
lock_word. */
(2)增加计数rw_lock_lock_word_incr(lock,X_LOCK_DECR) == X_LOCK_DECR,这时候需要向等待锁的线程发送信号:
if (lock->waiters) {
rw_lock_reset_waiter_flag(lock);
os_event_set(lock->event);
sync_array_object_signalled(sync_primary_wait_array);
}
os_event_set函数会发送一个pthread_cond_broadcast给等待的线程
4.监控读写锁
为了防止mysqld被hang住导致的长时间等待rw锁,error监控线程会对长时间等待的线程进行监控。这个线程每1秒loop一次
(os_event_wait_time_low(srv_error_event, 1000000, sig_count);)
函数入口:srv_error_monitor_thread
函数sync_array_print_long_waits()用于处理长时间等待信号量的线程,流程如下:
1. 查看sync_primary_wait_array数组中的所有等待线程。
->大于240秒时,向错误日志中输出警告,设置noticed = TRUE;
->大于600秒时,设置fatal =TRUE;
2.当noticed为true时,打印出innodb监控信息,然后sleep30秒
3. 返回fatal值
当函数sync_primary_wait_array返回true时,对于同一个等待线程还会有十次机会,也就是300 + 1*10(监控线程每次loop sleep 1s)秒的时间;如果挺不过去,监控线程就会执行一个断言失败:
if (fatal_cnt > 10) {
fprintf(stderr,
"InnoDB:Error: semaphore wait has lasted"
"> %lu seconds/n"
"InnoDB:We intentionally crash the server,"
"because it appears to be hung./n",
(ulong) srv_fatal_semaphore_wait_threshold);
ut_error;
}
ut_error是一个宏:
#define ut_error assert(0)
断言失败导致mysqld crash
在函数srv_error_monitor_thread里发现一个比较有意思的参数srv_kill_idle_transaction,对应的系统变量为innodb_kill_idle_transaction,用于清理在一段时间内的空闲事务。这个变量指定了空闲事务的最长时间。具体实现分析,且听下回分解
作者 记录成长之路 bitsCN.com

How to effectively monitor MySQL performance? Use tools such as mysqladmin, SHOWGLOBALSTATUS, PerconaMonitoring and Management (PMM), and MySQL EnterpriseMonitor. 1. Use mysqladmin to view the number of connections. 2. Use SHOWGLOBALSTATUS to view the query number. 3.PMM provides detailed performance data and graphical interface. 4.MySQLEnterpriseMonitor provides rich monitoring functions and alarm mechanisms.

The difference between MySQL and SQLServer is: 1) MySQL is open source and suitable for web and embedded systems, 2) SQLServer is a commercial product of Microsoft and is suitable for enterprise-level applications. There are significant differences between the two in storage engine, performance optimization and application scenarios. When choosing, you need to consider project size and future scalability.

In enterprise-level application scenarios that require high availability, advanced security and good integration, SQLServer should be chosen instead of MySQL. 1) SQLServer provides enterprise-level features such as high availability and advanced security. 2) It is closely integrated with Microsoft ecosystems such as VisualStudio and PowerBI. 3) SQLServer performs excellent in performance optimization and supports memory-optimized tables and column storage indexes.

MySQLmanagescharactersetsandcollationsbyusingUTF-8asthedefault,allowingconfigurationatdatabase,table,andcolumnlevels,andrequiringcarefulalignmenttoavoidmismatches.1)Setdefaultcharactersetandcollationforadatabase.2)Configurecharactersetandcollationfor

A MySQL trigger is an automatically executed stored procedure associated with a table that is used to perform a series of operations when a specific data operation is performed. 1) Trigger definition and function: used for data verification, logging, etc. 2) Working principle: It is divided into BEFORE and AFTER, and supports row-level triggering. 3) Example of use: Can be used to record salary changes or update inventory. 4) Debugging skills: Use SHOWTRIGGERS and SHOWCREATETRIGGER commands. 5) Performance optimization: Avoid complex operations, use indexes, and manage transactions.

The steps to create and manage user accounts in MySQL are as follows: 1. Create a user: Use CREATEUSER'newuser'@'localhost'IDENTIFIEDBY'password'; 2. Assign permissions: Use GRANTSELECT, INSERT, UPDATEONmydatabase.TO'newuser'@'localhost'; 3. Fix permission error: Use REVOKEALLPRIVILEGESONmydatabase.FROM'newuser'@'localhost'; then reassign permissions; 4. Optimization permissions: Use SHOWGRA

MySQL is suitable for rapid development and small and medium-sized applications, while Oracle is suitable for large enterprises and high availability needs. 1) MySQL is open source and easy to use, suitable for web applications and small and medium-sized enterprises. 2) Oracle is powerful and suitable for large enterprises and government agencies. 3) MySQL supports a variety of storage engines, and Oracle provides rich enterprise-level functions.

The disadvantages of MySQL compared to other relational databases include: 1. Performance issues: You may encounter bottlenecks when processing large-scale data, and PostgreSQL performs better in complex queries and big data processing. 2. Scalability: The horizontal scaling ability is not as good as Google Spanner and Amazon Aurora. 3. Functional limitations: Not as good as PostgreSQL and Oracle in advanced functions, some functions require more custom code and maintenance.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool

SublimeText3 English version
Recommended: Win version, supports code prompts!