We recently made some changes to our infrastructure and now I can't keep MySQL replication running. The slave complains of a corrupt binlog and resetting it doesn't help. I keep seeing entries like this in my master log:
BINLOG ' WFxKTRNJAAAAPwAAAKY/YwAAABsAAAAAAAEACHdlYmVkaTMwAA1QYXJ0bmVyQ29uZmlnAAQICA8P BC0AYwAA WFxKTRhJAAAAXAAAAAJAYwAAABsAAAAAAAEABP//8BoSAAAAAAAAggMAAAAAAAAJUEFDS0NPVU5U Azc3MfAaEgAAAAAAAIIDAAAAAAAACVBBQ0tDT1VOVAM3NzI= '/*!*/;
In the slave log, this just comes across as "Unknown Event" and the replication fails at that position.
Here is my my.cnf for the master:
[mysqld] datadir=/data/mysql-data socket=/tmp/mysql.sock user=mysql log-error=/var/log/mysqld.log binlog-do-db = db1 binlog-do-db = db2 binlog-do-db = db3 binlog-do-db = db4 binlog_format = 'MIXED' log-bin = /data/mysql-binlogs/mysql-bin.log server-id=73 report-host=thisserver.mydomain.com thread_cache_size = 30 key_buffer_size = 700M myisam_sort_buffer_size = 300M table_cache = 256 sort_buffer_size = 4M read_buffer_size = 1M innodb_data_home_dir = /data/mysql-data/ innodb_data_file_path = InnoDB:100M:autoextend set-variable = innodb_buffer_pool_size=500M set-variable = innodb_additional_mem_pool_size=10M set-variable = max_connections=500 innodb_log_group_home_dir = /data/mysql-data set-variable = innodb_log_file_size=20M set-variable = innodb_log_buffer_size=8M innodb_flush_log_at_trx_commit = 1
Does anybody know what might be causing the gibberish in my master binlogs?
Best Answer
Surprisingly, that's not gibberish.
That indeed appears at the top of binlogs whenever you do mysqlbinlog to a binary log generated using MySQL 5.1 and MySQL 5.5. You will not see that gibberish in binary logs for MySQL 5.0 and back.
This is why the start point for replication from an empty binary log is
This is good to remember if you do MySQL Replication where the Master if MySQL 5.1 and the slave is MySQL 5.0. This could present a really big headache.
Replication from Master using 5.0 and Slave using 5.1 works fine, not the other way around.(According to MySQL Documentation, it is generally not supported for 3 reasons: 1) Binary Log Format, 2) Row-based Replication, 3) SQL Incompatibility).
Anyway, do a mysqlbinlog on the offending binary log on the master. If the resulting dump produces gibberish in the middle of the dump (which I have seen a couple of times in my DBA career) you may have to skip to position 98 (MySQL 5.0) or 106 (MySQL 5.1) or 107 (MySQL 5.5) of the master's next binary log and start replicating from there (SOB :( you may need to use MAATKIT tools mk-table-checksum and mk-table-sync to reload master changes not on the slave [if you want to be a hero]; even worse, mysqldump the master and reload the slave and start replication totally over [if you don't want to be a hero])
If the mysqlbinlog of the master is completely readable after the top gibberish you saw, it is possible the master's binary log is fine but the relay log on the slave is corrupt (due to transmission/CRC errors). If that's the case, just reload the relay logs by issuing the CHANGE MASTER TO command as follows:
Where
You can get MMMM and PPPP by doing
SHOW SLAVE STATUS\G
and usingTry it out and let me know !!!
BTW running CHANGE MASTER TO command erases the slave's current relay logs and starts fresh.