MySQL主从失败 错误Got fatal error 1236解决方法

MySQL主从失败 错误Got fatal error 1236解决方法
由于主服务器异外重启, 导致从报错, 错误如下:
show slave status错误:

mysql> show slave status\G
Master_Log_File: mysql-bin.000288
Read_Master_Log_Pos: 627806304
Relay_Log_File: mysql-relay-bin.000990
Relay_Log_Pos: 627806457
Relay_Master_Log_File: mysql-bin.000288
Slave_IO_Running: No
Slave_SQL_Running: Yes
Exec_Master_Log_Pos: 627806304
Relay_Log_Space: 627806663

......
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log:
'Client requested master to start replication from impossible position'

mysql错误日志:

tail /data/mysql/mysql-error.log
111010 17:35:49 [ERROR] Error reading packet from server: Client requested master
to start replication from impossible position ( server_errno=1236)
111010 17:35:49 [ERROR] Slave I/O: Got fatal error 1236 from master when reading data
from binary log: 'Client requested master to start replication from impossible
position', Error_code: 1236
111010 17:35:49 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.000288',
position 627806304

按照习惯, 先尝试必改position位置.

mysql> stop slave;
mysql> change master to master_log_file='mysql-bin.000288',master_log_pos=627625751;
mysql> start slave;

错误依旧, 接下来登陆到主服务器查看binlog日志.
先按照错误点的标记去主服务器日志中查找:

[root@db1 ~]# mysqlbinlog --start-position=627655136 /data/mysql/binlog/mysql-bin.000288
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#111010 13:31:19 server id 4 end_log_pos 106 Start: binlog v 4, server v 5.1.45-log
created 111010 13:31:19
# Warning: this binlog is either in use or was not closed properly.
BINLOG '
F1aTTg8EAAAAZgAAAGoAAAABAAQANS4xLjQ1LWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
'/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

没有看到这个位置.

[root@db1 ~]# mysqlbinlog /data/mysql/binlog/mysql-bin.000288 > test.txt

less text.txt
看最后一部分
# at 627625495
#111010 16:35:46 server id 1 end_log_pos 627625631 Query thread_id=45613333
exec_time=32758 error_code=0
SET TIMESTAMP=1318289746/*!*/;
delete from freeshipping_bef_update where part='AR-4006WLM' and code=''
/*!*/;
# at 627625631
#111010 16:35:46 server id 1 end_log_pos 627625751 Query thread_id=45613333
exec_time=32758 error_code=0
SET TIMESTAMP=1318289746/*!*/;
delete from shippingFee_special where part='AR-4006WLM'
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

找到最接近错误标记627655136的一个position是627625631.

再回到slave机器上change master, 将postion指向这个位置.

mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)

mysql> change master to master_log_file='mysql-bin.000288',master_log_pos=627625631;
Query OK, 0 rows affected (0.06 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

再次查看

mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Queueing master event to the relay log
Master_Host: 192.168.21.105
Master_User: rep
Master_Port: 3306
Connect_Retry: 10
Master_Log_File: mysql-bin.000289
Read_Master_Log_Pos: 25433767
Relay_Log_File: mysql-relay-bin.000003
Relay_Log_Pos: 630
Relay_Master_Log_File: mysql-bin.000289
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

主从同步正常了, 同样的方法修复其它slave机器.

修复办法：
广州
slave stop;
change master to master_host='113.105.93.149', master_user='rep1', master_password='repdgaDATAII3', master_log_file='mysql-bin.000451', master_log_pos=76095939;
slave start;

深圳
slave stop;
change master to master_host='192.168.81.10', master_user='rep1', master_password='repdgaDATAII3', master_log_file='mysql-bin.000451', master_log_pos=76095939;
slave start;

故障点
reboot system boot 2.6.18-164.el5 Fri Jul 5 12:58 (01:56)

mysql-bin.000451 为故障点所对应的日志；
ll /data/mysqldata
-rw-rw---- 1 mysql mysql 76096015 Jul 5 12:54 mysql-bin.000451

master_log_pos=76095939 为故障点之前的pos；
[root@mysql1 tmp]# /app/soft/mysql/bin/mysqlbinlog mysql-bin.000451 > 130705-1254.log
[root@mysql1 tmp]# tail -f 130705-1254.log
'/*!*/;
# at 76095939
#130705 12:54:16 server id 1 end_log_pos 76096015 Query thread_id=391 exec_time=0 error_code=0
SET TIMESTAMP=1373000056/*!*/;
COMMIT
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

备注：在修复同步之前，确保slave服务器配置里面有 slave_skip_errors = 1062 跳过数据重复产生错误。