测试基于gtid模式主从复制下数据不一致的处理

1.基本信息

主库：

IP：10.16.24.107 port:3377

server-id = 1073377

data_dir:/data/MySQL/mysql3377/data/

base_dir:/usr/local/mysql

版本：mysql 5.6.29-log

binlog_format:ROW

从库：

IP：10.16.24.108 port:3377

server-id = 1083377

data_dir:/data/mysql/mysql3377/data/

base_dir:/usr/local/mysql

版本：mysql 5.6.29-log

binlog_format:ROW

2.在主库和从库上准备数据

主库上操作：

use lots

create table test (id int,name varchar(20))

engine=innodb DEFAULT CHARSET=utf8;

insert into test values (1,'zeng1'),(2,'zeng2'),(3,'zeng3');

从库上操作:

update test set name='zeng3-1' where name='zeng3';

查看主库test表上数据：

(product)root@localhost [lots]> select * from test;

+------+-------+

| id | name |

+------+-------+

| 1 | zeng1 |

| 2 | zeng2 |

| 3 | zeng3 |

+------+-------+

3 rows in set (0.00 sec)

查看从库test表数据：

(product)root@localhost [lots]> select * from test;

+------+---------+

| id | name |

+------+---------+

| 1 | zeng1 |

| 2 | zeng2 |

| 3 | zeng3-1 |

+------+---------+

3 rows in set (0.00 sec)

3.模拟数据不一致报错

删除主库上id=3的数据：

(product)root@localhost [lots]> delete from test where id=3;

Query OK, 1 row affected (0.00 sec)

(product)root@localhost [lots]> select * from test;

+------+-------+

| id | name |

+------+-------+

| 1 | zeng1 |

| 2 | zeng2 |

+------+-------+

2 rows in set (0.00 sec)

从库上查看test表的数据：

(product)root@localhost [lots]> select * from test;

+------+---------+

| id | name |

+------+---------+

| 1 | zeng1 |

| 2 | zeng2 |

| 3 | zeng3-1 |

+------+---------+

3 rows in set (0.00 sec)

从库test表上数据没有发生变化，主是什么原因？

查看从库状态：

(product)root@localhost [lots]> show slave status\G

*************************** 1. row ***************************

Slave_IO_State: Waiting for master to send event

Master_Host: 10.16.24.107

Master_User: repl

Master_Port: 3377

Connect_Retry: 60

Master_Log_File: mysql-bin.000017

Read_Master_Log_Pos: 2366

Relay_Log_File: relay-bin.000002

Relay_Log_Pos: 2278

Relay_Master_Log_File: mysql-bin.000017

Slave_IO_Running: Yes

Slave_SQL_Running: No

Replicate_Do_DB:

Replicate_Ignore_DB:

Replicate_Do_Table:

Replicate_Ignore_Table:

Replicate_Wild_Do_Table:

Replicate_Wild_Ignore_Table:

Last_Errno: 1032

Last_Error: Worker 3 failed executing transaction 'ae2d957d-f682-11e5-94fa-0050568a0bcb:8' at master log mysql-bin.000017, end_log_pos 2335; Could

not execute Delete_rows event on table lots.test; Can't find record in 'test', Error_code: 1032; handler error HA_ERR_END_OF_FILE; the event's master log mysql-

bin.000017, end_log_pos 2335

Skip_Counter: 0

Exec_Master_Log_Pos: 2068

Relay_Log_Space: 2774

Until_Condition: None

Until_Log_File:

Until_Log_Pos: 0

Master_SSL_Allowed: No

Master_SSL_CA_File:

Master_SSL_CA_Path:

Master_SSL_Cert:

Master_SSL_Cipher:

Master_SSL_Key:

Seconds_Behind_Master: NULL

Master_SSL_Verify_Server_Cert: No

Last_IO_Errno: 0

Last_IO_Error:

Last_SQL_Errno: 1032

Last_SQL_Error: Worker 3 failed executing transaction 'ae2d957d-f682-11e5-94fa-0050568a0bcb:8' at master log mysql-bin.000017, end_log_pos 2335; Could

not execute Delete_rows event on table lots.test; Can't find record in 'test', Error_code: 1032; handler error HA_ERR_END_OF_FILE; the event's master log mysql-

bin.000017, end_log_pos 2335

Replicate_Ignore_Server_Ids:

Master_Server_Id: 1073377

Master_UUID: ae2d957d-f682-11e5-94fa-0050568a0bcb

Master_Info_File: mysql.slave_master_info

SQL_Delay: 0

SQL_Remaining_Delay: NULL

Slave_SQL_Running_State:

Master_Retry_Count: 86400

Master_Bind:

Last_IO_Error_Timestamp:

Last_SQL_Error_Timestamp: 160428 20:06:42

Master_SSL_Crl:

Master_SSL_Crlpath:

Retrieved_Gtid_Set: ae2d957d-f682-11e5-94fa-0050568a0bcb:1-8

Executed_Gtid_Set: 1a3f7187-0b93-11e6-9e55-0050568a6e64:1-3,

ae2d957d-f682-11e5-94fa-0050568a0bcb:1-7

Auto_Position: 1

1 row in set (0.00 sec)

4.问题分析

上面发现问题如下：

a.从库数据没有同步

b.从库SQL线程状态为no

c.从库SQL线程报错

主库状态：

(product)root@localhost [lots]> show master status\G

*************************** 1. row ***************************

File: mysql-bin.000017

Position: 2366

Binlog_Do_DB:

Binlog_Ignore_DB:

Executed_Gtid_Set: ae2d957d-f682-11e5-94fa-0050568a0bcb:1-8

1 row in set (0.00 sec)

若是生产环境的操作，可能不知道删除前的数据是什么样的？这时就要去分析Binlog，检查删除的数据：

分析mysql-bin.000017:

[mysql@mvxl0782 logs]$ mysqlbinlog -vv mysql-bin.000017 --stop-position=2366

#160428 20:06:42 server id 1073377 end_log_pos 2335 CRC32 0x6c2da148 Delete_rows: table id 71 flags: STMT_END_F

BINLOG '

0vwhVx3hYBAAMwAAAL8IAACAABtkZWxldGUgZnJvbSB0ZXN0IHdoZXJlIGlkPTOwAvoB

0vwhVxPhYBAAMgAAAPEIAAAAAEcAAAAAAAEABGxvdHMABHRlc3QAAgMPAjwAA9DQyls=

0vwhVyDhYBAALgAAAB8JAAAAAEcAAAAAAAEAAgAC//wDAAAABXplbmczSKEtbA==

'/*!*/;

### DELETE FROM `lots`.`test`

### WHERE

### @1=3 /* INT meta=0 nullable=1 is_null=0 */

### @2='zeng3' /* VARSTRING(60) meta=60 nullable=1 is_null=0 */

# at 2335

#160428 20:06:42 server id 1073377 end_log_pos 2366 CRC32 0xe509dd97 Xid = 56

COMMIT/*!*/;

SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog *//*!*/;

DELIMITER ;

# End of log file

ROLLBACK /* added by mysqlbinlog */;

上面日志看到，有删除了delete from test where @1=3 and @2='zeng3' ，@1是id列，@2是name列，而从库上id=3的name值是zeng3-1，说明主从id=3的name值不一致，又由于主库同步到从

库的日志，在从库上面是一行行匹配，数据不致，所以从库的sql线程报错。

5.问题处理

从库上执行如下操作：

stop slave;

set gtid_next="ae2d957d-f682-11e5-94fa-0050568a0bcb:8"; ---跳过报错

begin;commit; ---执行一个空事务代替报错事务

set gtid_next="AUTOMATIC";

start slave;

查看从库状态，显示正常。

(product)root@localhost [lots]> show slave status\G

*************************** 1. row ***************************

Slave_IO_State: Waiting for master to send event

Master_Host: 10.16.24.107

Master_User: repl

Master_Port: 3377

Connect_Retry: 60

Master_Log_File: mysql-bin.000017

Read_Master_Log_Pos: 2366

Relay_Log_File: relay-bin.000003

Relay_Log_Pos: 448

Relay_Master_Log_File: mysql-bin.000017

Slave_IO_Running: Yes

Slave_SQL_Running: Yes

Replicate_Do_DB:

Replicate_Ignore_DB:

Replicate_Do_Table:

Replicate_Ignore_Table:

Replicate_Wild_Do_Table:

Replicate_Wild_Ignore_Table:

Last_Errno: 0

Last_Error:

Skip_Counter: 0

Exec_Master_Log_Pos: 2366

Relay_Log_Space: 3071

Until_Condition: None

Until_Log_File:

Until_Log_Pos: 0

Master_SSL_Allowed: No

Master_SSL_CA_File:

Master_SSL_CA_Path:

Master_SSL_Cert:

Master_SSL_Cipher:

Master_SSL_Key:

Seconds_Behind_Master: 0

Master_SSL_Verify_Server_Cert: No

Last_IO_Errno: 0

Last_IO_Error:

Last_SQL_Errno: 0

Last_SQL_Error:

Replicate_Ignore_Server_Ids:

Master_Server_Id: 1073377

Master_UUID: ae2d957d-f682-11e5-94fa-0050568a0bcb

Master_Info_File: mysql.slave_master_info

SQL_Delay: 0

SQL_Remaining_Delay: NULL

Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it

Master_Retry_Count: 86400

Master_Bind:

Last_IO_Error_Timestamp:

Last_SQL_Error_Timestamp:

Master_SSL_Crl:

Master_SSL_Crlpath:

Retrieved_Gtid_Set: ae2d957d-f682-11e5-94fa-0050568a0bcb:1-8

Executed_Gtid_Set: 1a3f7187-0b93-11e6-9e55-0050568a6e64:1-3,

ae2d957d-f682-11e5-94fa-0050568a0bcb:1-8

Auto_Position: 1

1 row in set (0.00 sec)

再手工删除从库上test表id=3的记录：

(product)root@localhost [lots]> delete from test where id=3;

Query OK, 1 row affected (0.00 sec)

最后查看从库test表的记录，已与主库记录保持一致：

(product)root@localhost [lots]> select * from test;

+------+-------+

| id | name |

+------+-------+

| 1 | zeng1 |

| 2 | zeng2 |

+------+-------+

2 rows in set (0.00 sec)