全局读锁
全局锁就是对整个数据库加锁。是数据库实例层级的锁,MySQL提供了一个加全局读锁语句:Flush tables with read lock;
如果需要让整个库处于只读状态的时候,可以使用这个语句,在其之后执行的其他线程的DML,DDL语句都会被阻塞。
谁持有全局读锁?
会话1:
mysql> flush table with read lock;
Query OK, 0 rows affected (0.00 sec)
会话2:
mysql> select * from sbtest.sbtest1 limit 1;
+----+------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| id | k | c | pad |
+----+------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| 1 | 4993 | 83868641912-28773972837-60736120486-75162659906-27563526494-20381887404-41576422241-93426793964-56405065102-33518432330 | 67847967377-48000963322-62604785301-91415491898-96926520291 |
+----+------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> update sbtest.sbtest1 set pad='xxx' where id =1;
//DML被阻塞
会话3进行性能问题排查,没有任何有效信息:
mysql> select * from information_schema.innodb_trx;
Empty set (0.00 sec)
mysql> select * from sys.innodb_lock_waits;
Empty set (0.00 sec)
mysql> show engine innodb status\G
...
------------
TRANSACTIONS
------------
Trx id counter 3873557
Purge done for trx's n:o < 3873542 undo n:o < 0 state: running but idle
History list length 0
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 422068296891128, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296890272, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296889416, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296888560, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296887704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296886848, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422068296885992, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
会话4:
通过以上的常规手段没有任何用处,有GDB经验的人会开始使用gdb、strace、pstack等命令查看mysql的栈线程信息。
MySQL在5.7开始提供一个performance_schema.metadata_locks显示各种Server层的锁信息(包括全局读锁和DML锁信息等)。
SQL语句中, owner_thread_id != sys.ps_thread_id(connection_id())表示非本连接的其他连接。
mysql> select * from performance_schema.metadata_locks where owner_thread_id != sys.ps_thread_id(connection_id())\G
*************************** 1. row ***************************
OBJECT_TYPE: GLOBAL
OBJECT_SCHEMA: NULL
OBJECT_NAME: NULL
COLUMN_NAME: NULL
OBJECT_INSTANCE_BEGIN: 140592265233248
LOCK_TYPE: INTENTION_EXCLUSIVE
LOCK_DURATION: STATEMENT
LOCK_STATUS: PENDING
SOURCE: sql_base.cc:2993
OWNER_THREAD_ID: 65
OWNER_EVENT_ID: 5
*************************** 2. row ***************************
OBJECT_TYPE: GLOBAL
OBJECT_SCHEMA: NULL
OBJECT_NAME: NULL
COLUMN_NAME: NULL
OBJECT_INSTANCE_BEGIN: 140592008273008
LOCK_TYPE: SHARED
LOCK_DURATION: EXPLICIT
LOCK_STATUS: GRANTED
SOURCE: lock.cc:1032
OWNER_THREAD_ID: 59
OWNER_EVENT_ID: 43
*************************** 3. row ***************************
OBJECT_TYPE: COMMIT
OBJECT_SCHEMA: NULL
OBJECT_NAME: NULL
COLUMN_NAME: NULL
OBJECT_INSTANCE_BEGIN: 140591997514192
LOCK_TYPE: SHARED
LOCK_DURATION: EXPLICIT
LOCK_STATUS: GRANTED
SOURCE: lock.cc:1107
OWNER_THREAD_ID: 59
OWNER_EVENT_ID: 43
通过performance_schema.metadata_locks排查谁有全局读锁。
全局读锁在该表中通常记录中同一个会话的OBJECT_TYPE为GLOBAL和COMMIT,LOCK_TYPE都为SHARED的两把显式锁。
看到第一行:
LOCK_STATUS: PENDING表示正在等在被授予。
OWNER_THREAD_ID: 65表示被阻塞的内部线程id为65。
#使用会话4继续查询:
mysql> show processlist;
+----+-----------------+-----------------------+--------------------+---------+--------+------------------------------+-------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+-----------------------+--------------------+---------+--------+------------------------------+-------------------------------------------------+
| 5 | event_scheduler | localhost | NULL | Daemon | 321879 | Waiting on empty queue | NULL |
| 21 | root | localhost | xpp | Sleep | 519 | | NULL |
| 25 | root | 180.167.153.202:57512 | NULL | Sleep | 496 | | NULL |
| 26 | root | 180.167.153.202:57513 | performance_schema | Sleep | 496 | | NULL |
| 27 | root | localhost | NULL | Query | 1100 | Waiting for global read lock | update sbtest.sbtest1 set pad='xxx' where id =1 |
| 28 | root | localhost | NULL | Sleep | 965 | | NULL |
| 29 | root | localhost | NULL | Query | 0 | starting | show processlist |
+----+-----------------+-----------------------+--------------------+---------+--------+------------------------------+-------------------------------------------------+
mysql> select sys.ps_thread_id(21);
+----------------------+
| sys.ps_thread_id(21) |
+----------------------+
| 59 |
+----------------------+
1 row in set (0.00 sec)
mysql> select sys.ps_thread_id(27);
+----------------------+
| sys.ps_thread_id(27) |
+----------------------+
| 65 |
+----------------------+
1 row in set (0.00 sec)
##使用
##select a.thread_id,b.processlist_id,a.SQL_text from performance_schema.events_statements_current a join performance_schema.threads b on a.thread_id=b.thread_id;
##替代select sys.ps_thread_id(27);会更有效
可以看到:
connection_id为21的连接,内部线程号为59,对应为持有全局锁的线程。
connection_id为27的连接,内部线程号为65,对应为等待锁的线程。
此时可以通过kill掉21号线程解锁。
mysql> kill 21;
Query OK, 0 rows affected (0.00 sec)
##此时 拥有全局读锁的连接被杀死
##此时 解锁了全剧读锁,所以connection_id为27的连接,内部线程号为65,可以顺利执行update,也执行完毕释放了锁。
##此时 performance_schema.metadata_locks为空。
mysql> select * from performance_schema.metadata_locks where owner_thread_id != sys.ps_thread_id(connection_id())\G
Empty set (0.00 sec)
DML锁
metadata lock即MDL,是⽤于保护MySQL内部对象的元数据,MySQL通过MDL保护DDL和DML的并发。
MDL在MySQL5.5版本引⼊,在此之前MySQL对于元数据的保护仅仅是语句级别的,引⼊MDL后,MySQL对于元数据的保护上升为事务级别的。
谁持有DML锁?
准备工作
会话1:
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update sbtest.sbtest1 set pad='xxx' where id =3;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
##执行update但是不提交
会话2:
mysql> alter table sbtest.sbtest1 add index i_c(C);
##被阻塞
会话3分析性能问题:
mysql> show processlist;
+----+-----------------+-----------------------+--------------------+---------+--------+---------------------------------+---------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+-----------------------+--------------------+---------+--------+---------------------------------+---------------------------------------------+
| 5 | event_scheduler | localhost | NULL | Daemon | 323216 | Waiting on empty queue | NULL |
| 25 | root | 180.167.153.202:57512 | NULL | Sleep | 30 | | NULL |
| 26 | root | 180.167.153.202:57513 | performance_schema | Sleep | 30 | | NULL |
| 31 | root | localhost | NULL | Query | 22 | Waiting for table metadata lock | alter table sbtest.sbtest1 add index i_c(C) |
| 32 | root | localhost | NULL | Query | 0 | starting | show processlist |
| 33 | root | localhost | NULL | Sleep | 53 | | NULL |
+----+-----------------+-----------------------+--------------------+---------+--------+---------------------------------+---------------------------------------------+
6 rows in set (0.00 sec)
额外提一个点,假设此时有会话4,会话5,会话6,会话7…查询这张表:
mysql> select * from sbtest.sbtest1;
mysql> select id from sbtest.sbtest1;
## 如果越多的人在此时执行查这张表,就会累积越多的Waiting for table metadata lock ,这会造成MySQL阻塞或最终崩溃
会话3继续分析故障问题:
方法一:
##查看执行完成但是没提交的表:
mysql> select d.trx_started ,a.thread_id,b.processlist_id,a.SQL_text from performance_schema.events_statements_current a join performance_schema.threads b on a.thread_id=b.thread_id join information_schema.processlist c on b.processlist_id=c.id join information_schema.innodb_trx d on c.id=d.trx_mysql_thread_id order by d.trx_started;
+---------------------+-----------+----------------+-------------------------------------------------+
| trx_started | thread_id | processlist_id | SQL_text |
+---------------------+-----------+----------------+-------------------------------------------------+
| 2020-09-04 11:59:30 | 75 | 37 | update sbtest.sbtest1 set pad='xxx' where id =3 |
+---------------------+-----------+----------------+-------------------------------------------------+
1 row in set (0.00 sec)
可以看到37号连接执行了update并没提交或者回滚。
方法二:
在MySQL5.7之前,不能直观的看到谁持有MDL锁,(除非使用GDB),现在可以查performance_schema.metadata_locks得知MDL的消息。select a.thread_id,b.processlist_id,a.SQL_text from performance_schema.events_statements_current a join performance_schema.threads b on a.thread_id=b.thread_id;
select * from performance_schema.metadata_locks where owner_thread_id != sys.ps_thread_id(connection_id());
可以看到第一行, sbtest.btest1表的 SHARED_WRITE 锁 GRANTED状态表示被持有, 对应的内部线程为75,连接号是37。
下面7行都是内部线程76(连接号38)的状态,其中SHARED_UPGRADABLE在 GRANTED状态,EXCLUSIVE在 PENDING的状态。表示38号连接在等待MDL锁。
DML锁总结
这里我认为值用方法二进行查询能够完全了解这些SQL在做什么内容的操作,并且查出来MDL的锁是被谁持有的。很好用的方法。
表级锁
表级锁是对当前操作的整张表加锁,MYISAM与INNODB都支持表级锁定。表级锁定分为表共享读锁(共享锁)与表独占写锁(排他锁)。
谁持有表级锁?
会话1:
mysql> use sbtest;
mysql> lock table sbtest1 read;
Query OK, 0 rows affected (0.00 sec)
会话2:
mysql> update sbtest.sbtest1 set pad='xxx' where id =3;
##被阻塞
会话3:
可以发现,update语句在等待MDL锁。但是sleep线程的SQL语句无法确定。
既然是MDL锁,即查看performance_schema.metadata_locks
select a.thread_id,b.processlist_id,a.SQL_text from performance_schema.events_statements_current a join performance_schema.threads b on a.thread_id=b.thread_id;
select * from performance_schema.metadata_locks where owner_thread_id != sys.ps_thread_id(connection_id());
查看information_schema.innodb_trx,sys.innodb_lock_waits的内容,是否有记录,发现是查不到任何有用信息的:
mysql> select * from information_schema.innodb_trx;
Empty set (0.00 sec)
mysql> select * from sys.innodb_lock_waits;
Empty set (0.00 sec)
我们可以用过查询表级别的锁信息(performance_schema.table_handles):
mysql> select * from performance_schema.table_handles where owner_thread_id != 0 \G
*************************** 1. row ***************************
OBJECT_TYPE: TABLE
OBJECT_SCHEMA: sbtest
OBJECT_NAME: sbtest1
OBJECT_INSTANCE_BEGIN: 140592197772056
OWNER_THREAD_ID: 79
OWNER_EVENT_ID: 17
INTERNAL_LOCK: NULL
EXTERNAL_LOCK: READ EXTERNAL
1 row in set (0.00 sec)
可以看到,内部id为79(连接号41)对sbtest.sbtest1表加了表级读锁,结合processlist可以中和掉长时间处于sleep状态,所以update语句一直在等待读锁。
表级锁总结
此时可以和开发确认,如果没有什么特殊操作,可以尝试杀死这个线程,同时针对问题进行优化,避免再发生类似的情况。
行级锁
performance_schema中data_lock是MySQL8.0中新增的。
如果一个事务长时间没有提及,虽然可以从information_schema.innodb_trx,performance_schema.events_statements_current等表中查询到相应的事物信息,但是不知道这个事务的持锁信息的。虽然information_schema.innodb_locks用于记录事务的锁信息,但需要在两个不同事务发生锁等待时该表才会记录下来两个事务的锁信息。从MySQL8.0开始,在performance_schema中存在data_locks表记录任意事务的锁信息(同时废弃了information_schema.innodb_locks表),不需要有锁等待关系存在(注意,该表中只记录innodb存储引擎层的锁)。
会话1:
mysql> create table a111(id int,test int,datet_time timestamp );
mysql> alter table xpp.a111 add primary key(id);
mysql> insert into a111 values(2,1,now());
mysql> select * from a111;
+------+------+---------------------+
| id | test | datet_time |
+------+------+---------------------+
| 2 | 1 | 2020-09-04 17:46:23 |
+------+------+---------------------+
1 row in set (0.00 sec)
mysql> set autocommit=off;
mysql> begin;
mysql> update xpp.a111 set datet_time =now() where id =2;
会话2:
mysql> select * from performance_schema.data_locks\G
*************************** 1. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161409432:1104:139970165682704
ENGINE_TRANSACTION_ID: 3874665
THREAD_ID: 97
EVENT_ID: 24
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: NULL
OBJECT_INSTANCE_BEGIN: 139970165682704
LOCK_TYPE: TABLE
LOCK_MODE: IX
LOCK_STATUS: GRANTED
LOCK_DATA: NULL
*************************** 2. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161409432:47:4:2:139970165679600
ENGINE_TRANSACTION_ID: 3874665
THREAD_ID: 97
EVENT_ID: 24
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: PRIMARY
OBJECT_INSTANCE_BEGIN: 139970165679600
LOCK_TYPE: RECORD
LOCK_MODE: X,REC_NOT_GAP
LOCK_STATUS: GRANTED
LOCK_DATA: 2
2 rows in set (0.00 sec)
##LOCK_DATA: 2被锁定的数据记录,这里的记录对应的是INDEX_NAME: PRIMARY的value
查询结果中,两行锁记录,一行是对表xpp的IX锁,状态为GRANTED,另一个锁为主键索引的X锁,REC_NOT_GAP我理解为 lock_mode x locks rec but not gap 。
如果没有创建primary key,会出现 INDEX_NAME:gen_clust_index 取代 INDEX_NAME: PRIMARY,如果表没有主键或唯一索引InnoDB内部适用,生成一个隐藏的聚集索引为合成列包含行ID值gen_clust_index。
现在,模拟两条DML发生锁等待的场景:
在会话1未提交的状态下,开启新的会话3:mysql> update xpp.a111 set datet_time =now() where id =2;
此时查看表:
mysql> select * from performance_schema.data_locks\G
*************************** 1. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161410288:1104:139970165688864
ENGINE_TRANSACTION_ID: 3874666
THREAD_ID: 99
EVENT_ID: 7
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: NULL
OBJECT_INSTANCE_BEGIN: 139970165688864
LOCK_TYPE: TABLE
LOCK_MODE: IX
LOCK_STATUS: GRANTED
LOCK_DATA: NULL
*************************** 2. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161410288:47:4:2:139970165685760
ENGINE_TRANSACTION_ID: 3874666
THREAD_ID: 99
EVENT_ID: 7
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: PRIMARY
OBJECT_INSTANCE_BEGIN: 139970165685760
LOCK_TYPE: RECORD
LOCK_MODE: X,REC_NOT_GAP
LOCK_STATUS: WAITING
LOCK_DATA: 2
*************************** 3. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161409432:1104:139970165682704
ENGINE_TRANSACTION_ID: 3874665
THREAD_ID: 97
EVENT_ID: 24
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: NULL
OBJECT_INSTANCE_BEGIN: 139970165682704
LOCK_TYPE: TABLE
LOCK_MODE: IX
LOCK_STATUS: GRANTED
LOCK_DATA: NULL
*************************** 4. row ***************************
ENGINE: INNODB
ENGINE_LOCK_ID: 139970161409432:47:4:2:139970165679600
ENGINE_TRANSACTION_ID: 3874665
THREAD_ID: 97
EVENT_ID: 24
OBJECT_SCHEMA: xpp
OBJECT_NAME: a111
PARTITION_NAME: NULL
SUBPARTITION_NAME: NULL
INDEX_NAME: PRIMARY
OBJECT_INSTANCE_BEGIN: 139970165679600
LOCK_TYPE: RECORD
LOCK_MODE: X,REC_NOT_GAP
LOCK_STATUS: GRANTED
LOCK_DATA: 2
4 rows in set (0.00 sec)
这四行记录中,新增了两条线程id为99的记录,IX的表锁状态GRANTED,X的行数状态是WAITING。说明正在等待锁被授予。这里并不能很直观的查到锁等待关系。利用sys.innodb_lock_waits查一下:
(MySQL5.7中,可以使用sys.innodb_lock_waits查询,MySQL8.0中也可以,额外说明的是8.0中,该视图的查询表发生变化,由information_schema.innodb_locks,information_schema.innodb_locks_waits 变成 performance_schema.data_locks, performance_schema.data_locks_waits )
mysql> select * from sys.innodb_lock_waits\G
*************************** 1. row ***************************
wait_started: 2020-09-08 14:58:13
wait_age: 00:00:29
wait_age_secs: 29
locked_table: `xpp`.`a111`
locked_table_schema: xpp
locked_table_name: a111
locked_table_partition: NULL
locked_table_subpartition: NULL
locked_index: PRIMARY
locked_type: RECORD
waiting_trx_id: 3874666
waiting_trx_started: 2020-09-08 14:58:13
waiting_trx_age: 00:00:29
waiting_trx_rows_locked: 1
waiting_trx_rows_modified: 0
waiting_pid: 61
waiting_query: update xpp.a111 set datet_time =now() where id =2
waiting_lock_id: 139970161410288:47:4:2:139970165685760
waiting_lock_mode: X,REC_NOT_GAP
blocking_trx_id: 3874665
blocking_pid: 59
blocking_query: NULL
blocking_lock_id: 139970161409432:47:4:2:139970165679600
blocking_lock_mode: X,REC_NOT_GAP
blocking_trx_started: 2020-09-08 14:50:07
blocking_trx_age: 00:08:35
blocking_trx_rows_locked: 1
blocking_trx_rows_modified: 1
sql_kill_blocking_query: KILL QUERY 59
sql_kill_blocking_connection: KILL 59
1 row in set (0.00 sec)