MySQL – Error 1236: Could Not Find First Log File Name in Binary Log Index File

mariadbMySQLreplication

Our setup:

Master: MariaDB 10.0.21
Slave: MariaDB 10.0.17

Replication was working fine until recently at which point the slave's DBs had to be restored from a dump. I performed all of the necessary steps: Dump the master's DBs, transfer the dump to the slave, drop the old DBs, execute the dump to restore the DBs, execute the appropriate CHANGE MASTER command, and finally START SLAVE.

I am receiving the error:
Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

The first log file that the slave needs from the master is mysql-bin.000289. I can see that this is present on the master:

I can also see that the binary log index on the master seems to have an entry for this log file:

Still replication is not working – I keep getting the same error. I'm out of ideas – what should I check next?

Updated: Output of SHOW SLAVE STATUS\G as requested:

MariaDB [(none)]> SHOW SLAVE STATUS\G
--------------
SHOW SLAVE STATUS
--------------

*************************** 1. row ***************************
               Slave_IO_State: 
                  Master_Host: 127.0.0.1
                  Master_User: replication
                  Master_Port: 1234
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000289
          Read_Master_Log_Pos: 342
               Relay_Log_File: mysqld-relay-bin.000002
                Relay_Log_Pos: 4
        Relay_Master_Log_File: mysql-bin.000289
             Slave_IO_Running: No
            Slave_SQL_Running: Yes
              Replicate_Do_DB: xxx_yyy,xxx_zzz
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 342
              Relay_Log_Space: 248
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 1236
                Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 3
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
                   Using_Gtid: No
                  Gtid_IO_Pos: 
1 row in set (0.00 sec)

Additional requested information:

root@master [818 18:54:22 /var/lib/mysql]# ls -l /var/lib/mysql/mysql-bin.000289
-rw-rw---- 1 mysql mysql 1074010194 May 19 03:28 /var/lib/mysql/mysql-bin.000289
root@master [819 18:54:29 /var/lib/mysql]# ls mysql-bin.00029*
mysql-bin.000290  mysql-bin.000291  mysql-bin.000292 #(Yes, it was created)
root@master [821 18:56:52 /var/lib/mysql]# mysql -uroot -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6345382
Server version: 10.0.21-MariaDB-log MariaDB Server

Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SHOW BINARY LOGS;
+------------------+------------+
| Log_name         | File_size  |
+------------------+------------+
| mysql-bin.000279 | 1074114047 |
| mysql-bin.000280 | 1074004090 |
| mysql-bin.000281 | 1074035416 |
| mysql-bin.000282 | 1073895128 |
| mysql-bin.000283 | 1073742000 |
| mysql-bin.000284 | 1074219591 |
| mysql-bin.000285 | 1074184547 |
| mysql-bin.000286 | 1074217812 |
| mysql-bin.000287 | 1022733058 |
| mysql-bin.000288 |     265069 |
| mysql-bin.000289 | 1074010194 |
| mysql-bin.000290 | 1074200346 |
| mysql-bin.000291 |  617421886 |
| mysql-bin.000292 |     265028 |
+------------------+------------+
14 rows in set (0.00 sec)

MariaDB [(none)]> exit
Bye
root@master [821 18:57:24 /var/lib/mysql]# mysqlbinlog mysql-bin.000289 > /tmp/somefile.txt
root@master [822 18:58:13 /var/lib/mysql]# tail /tmp/somefile.txt 
# at 1074010124
#160519  3:28:59 server id 5  end_log_pos 1074010151    Xid = 417608063
COMMIT/*!*/;
# at 1074010151
#160519  3:28:59 server id 5  end_log_pos 1074010194    Rotate to mysql-bin.000290  pos: 4
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
root@master [823 18:58:31 /var/lib/mysql]#

/etc/my.cnf.d/server.cnf (excerpt):

# BINARY LOGGING #
log-bin                        = /var/lib/mysql/mysql-bin
expire-logs-days               = 14
sync-binlog                    = 1

Edit: Postion 342 does seem to exist:

root@master [826 12:15:33 /var/lib/mysql]# grep "end_log_pos 342 " /tmp/somefile.txt
#160517 14:43:13 server id 5  end_log_pos 342   Binlog checkpoint mysql-bin.000288

Best Answer

You seem to not be connecting to the master that you think you are. Per your binary logs on the master you seem to have:

#160519 3:28:59 server id 5

But per SHOW SLAVE STATUS we see:

         Master_Server_Id: 3

And further you seem to be connecting on localhost, but you implied your master/slave are on different hosts:

              Master_Host: 127.0.0.1

Related Solutions

Mysql – Looking for an efficient way to fix “Could not parse relay log event entry…” error

There is more stable approach you can try

Here is something to remember

Whenever you run CHANGE MASTER TO, it will erase every relay log you have. You do not want to keep relay logs of commands you have not executed any SQL on as of yet

The following is an excerpt taken from a post I made back on Feb 03, 2012 : How to resolve the master server shut down/unavailability in mysql with master - slave replication :

Please notice that there are two sets of replication coordinates from the Master

(Master_Log_File,Read_Master_Log_Pos)

(Relay_Master_Log_File,Exec_Master_Log_Pos)

There is a major difference between them

(Master_Log_File,Read_Master_Log_Pos) tells you the last binlog statement from the Master's log file and log position that the Slave read from the Master and placed in its Relay Logs.

(Relay_Master_Log_File,Exec_Master_Log_Pos) tells you the last binlog statement from the Master's log file and log position that the Slave read from the Master and placed in its Relay Logs THAT IS NEXT TO BE EXECUTED ON THE SLAVE.

What you want are two things:

Erase Every Binary Log You Have
Start Collecting Binary Log Entries From the Last SQL You Successfully Executed.

In your case, you must use the second set of Replication Coordinates

Relay_Master_Log_File
Exec_Master_Log_Pos

It is easy to distrust a corrupt relay log as shown in the error message. The one that hurts the most is a corrupt Master Log. You will have to jump through hoops if that is the case. On the other hand, if one of the other situations was the reason for the corrupt relay log, the simplest and most concise approach is what I stated.

To make sure, whatever is reported for Relay_Master_Log_File, if that particular binary log still exists on the Master, perform a mysqlbinlog on it. If it dumps in its entirety without corrupt characters, go ahead and use the second set of replication coordinates.

From my same earlier post

mysql> show slave status\G
*************************** 1. row ***************************
             Slave_IO_State: Waiting for master to send event
                Master_Host: 10.48.20.253
                Master_User: replicant
                Master_Port: 3306
              Connect_Retry: 60
            Master_Log_File: mysql-bin.000254
        Read_Master_Log_Pos: 858190247
             Relay_Log_File: relay-bin.066069
              Relay_Log_Pos: 873918
      Relay_Master_Log_File: mysql-bin.000254
           Slave_IO_Running: Yes
          Slave_SQL_Running: Yes
            Replicate_Do_DB:
        Replicate_Ignore_DB:
         Replicate_Do_Table:
     Replicate_Ignore_Table:
    Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
                 Last_Errno: 0
                 Last_Error:
               Skip_Counter: 0
        Exec_Master_Log_Pos: 858190247
            Relay_Log_Space: 873772
            Until_Condition: None
             Until_Log_File:
              Until_Log_Pos: 0
         Master_SSL_Allowed: No
         Master_SSL_CA_File:
         Master_SSL_CA_Path:
            Master_SSL_Cert:
          Master_SSL_Cipher:
             Master_SSL_Key:
      Seconds_Behind_Master: 0
1 row in set (0.00 sec)

notice that the Replication Coordinates from SHOW SLAVE STATUS\G for what was last executed are (mysql-bin.000254,858190247). The CHANGE MASTER TO command in this case would be:

CHANGE MASTER TO master_log_file='mysql-bin.000254',master_log_pos=858190247;

Give it a Try !!!

UPDATE 2012-09-14 16:38 EDT

If you worried about the stockpiling relay logs, just throttle the relay logs. In SHOW SLAVE STATUS\G, there is a field called Relay_Log_Space. That gives you the sum of all relay sizes in bytes. Did you know you could put a cap on that number ?

The option is called relay_log_space_limit.

For example, if you want to cap the total number of bytes to 10G, do the following

STEP 01) Add this to /etc/my.cnf on the Slave

[mysqld]
relay_log_space_limit = 10G

STEP 02) Run service mysql restart on the Slave

and that's it !!!

When the oldest relay has all its entries processed, it is deleted and a new relay log is created. That gets filled until all relay logs add up to 10G. That's the only way to control runaway relay log space issues.

UPDATE 2012-09-14 18:10 EDT

SUGGESTION : If you make mysqldump backups of the data on the Slave every midnight, you could set up the following to restrict having 1TB of binary logs:

STEP 01) Add this to /etc/my.cnf on the Master

[mysqld]
expire_logs_days = 14

STEP 02) Run this query on the Master

mysql> PURGE BINARY LOGS BEFORE DATE(NOW()) - INTERVAL 14 DAY;

STEP 03) service mysql restart on the Master

STEP 04) Add a mysqldump backup script to a crontab on the Slave

This will make the Slave more useful and would control having excess binary logs to worry about

Mysql – Broken thesql replication on slave server (error 1236)

I believe the log file and position need to be specified with the host.

Try:

CHANGE MASTER TO MASTER_HOST='192.168.1.105', MASTER_USER='slaveuser', MASTER_PASSWORD='mypassword', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=107;