MySQL 5.7复制延迟之sync_relay_log
一、描述
MySQL 5.7版本主从复制,批量时候显示延迟上万秒。
二、现象
1、io使用率高
#iostat -dxm 1 1000
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
scd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vdb 0.00 96.00 0.00 2596.00 0.00 8.54 6.74 1.33 0.51 0.37 95.30
vdc 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vdd 0.00 0.00 0.00 11.00 0.00 0.06 11.64 0.00 0.09 0.09 0.10
vde 0.00 0.00 0.00 7.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vdf 0.00 0.00 0.00 511.00 0.00 0.00 0.00 0.05 0.09 0.09 4.60
vdg 0.00 0.00 0.00 511.00 0.00 0.00 0.00 0.05 0.09 0.09 4.80
dm-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-2 0.00 0.00 0.00 34.00 0.00 0.23 13.65 0.02 0.59 0.38 1.30
dm-3 0.00 0.00 0.00 2144.00 0.00 8.38 8.00 1.40 0.65 0.45 97.20
dm-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
2、dm3是relay log 和binlog分区
$ ls -l /dev/mapper
total 0
lrwxrwxrwx 1 root root 7 Jul 23 23:20 backup-backup -> ../dm-0
crw-rw---- 1 root root 10, 58 Jul 23 23:20 control
lrwxrwxrwx 1 root root 7 Jul 23 23:20 VG00-lv_root -> ../dm-4
lrwxrwxrwx 1 root root 7 Jul 23 23:20 zxmysql-zxdba -> ../dm-1
lrwxrwxrwx 1 root root 7 Jul 23 23:20 zxmysql-zxlog -> ../dm-3
3、slave状态
mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Queueing master event to the relay log
略.........................................
Connect_Retry: 60
Master_Log_File: mysql-bin.011494
Read_Master_Log_Pos: 21037034
Relay_Log_File: relay-log.001904
Relay_Log_Pos: 3154097
Relay_Master_Log_File: mysql-bin.011494
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 3153884
Relay_Log_Space: 21037535
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 471
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 400011
Master_UUID: 0f8507ea-6da1-11e8-8646-005056873c4a
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Reading event from the relay log
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 0f8507ea-6da1-11e8-8646-005056873c4a:14137114-19288497
Executed_Gtid_Set: 0f8507ea-6da1-11e8-8646-005056873c4a:1-19288446
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.01 sec)
ERROR:
No query specified
三、分析
通过以上现象发现备库io使用率过高,超过90%。io过高的磁盘为日志盘,存放relay log和binlog。io thead一致在写relay log,调用fdatasync写磁盘。这里涉及到一个参数sync_relay_log,默认值为10000,查看当前系统参数值为1.
四、解决方案
优化io thread线程和sql thread线程。sync_relay_log使用默认值,使用mts优化sql thread。
stop slave;
set global slave_parallel_type=logical_clock;
set global slave_parallel_workers=8;
set global sync_master_info=10000;
set global sync_relay_log=10000;
set global sync_relay_log_info=10000;
start slave;
Linux公社的RSS地址:https://www.linuxidc.com/rssFeed.aspx
本文永久更新链接地址:https://www.linuxidc.com/Linux/2019-09/160840.htm
- MySQL 5.7复制延迟之sync_relay_log
- 解决Mysql复制Relay log read failure 的问题
- mysql-5.7 调整mysql的复制方式由master_log_file+master_log_pos 到gtid 详解
- mysql复制的一例错误:Relay_Log_Pos不变
- [MySQL] 号称永久解决了复制延迟问题的并行复制,MySQL5.7
- MySQL复制报错(Slave failed to initialize relay log info structure from the repository)
- MySQL:延迟的主从复制 ( Delayed Replication )
- Mysql 5.7 基于组复制(MySQL Group Replication) - 运维小结
- mysql 5.7 To log in you must change it using a client that supports expired passwords
- MySQL延迟主从复制
- MySQL主从复制延迟的监测及缓解
- MySQL5.7新特性之Multi-Source多源复制
- MySQL 主从延迟复制方法总结
- MySQL异步复制备忘(主从复制,GR的基础,延迟问题参见mysql-tranfer)
- pt-heartbeat监控mysql主从复制延迟
- mysql5.7密码过期ERROR 1862 (HY000): Your password has expired. To log in you must change
- 【MySQL案例】error.log的Warning:If a crash happens thisconfiguration does not guarantee that the relay lo(转)
- MySQL Slave relay_log损坏修复
- 【MySQL】sync_binlog innodb_flush_log_at_trx_commit 浅析
- MySQL Slave relay_log损坏修复