Check the MySQL error log. It looks like trying to read table 44 is actually crashing the server.
"Lost connection to MySQL server during query" is (or should be) heart-stopping time for a MySQL DBA because it often means that whatever your query just did has actually crashed the server.
The subsequent messages seem to bear this out:
mysqldump: Got error: 2003: Can't connect to MySQL server on 'localhost' (10061) when
trying to connect
Operation failed with exitcode 2
A quick test of mysqldump on Windows 7 against a MySQL server that is not running returns exactly that same error and exits with %ERRORLEVEL%
set to 2. The nature of error 10061 can be found with the perror
utility.
C:\>perror 10061
Win32 error code 10061: No connection could be made because the target machine actively refused it.
...so, check the MySQL error log. What's likely happening is that you're hitting corruption severe enough that in spite of innodb_force_recovery
, the MySQL Server is crashing. I will then usually restart on its own, but this takes time so may not complete before mysqldump gets an IP connection refused on the subsequent connection attempts. This should all show up in the error log.
I assume that when the dump does not yield any error, the table's data is non-corrupted.
Incorrect assumption. If the dump does not yield any error, it could also mean that that innodb_force_recovery
allowed the server to retrieve some of the data. A "successful" dump of a table only means it's not corrupted so severely that innodb_force_recovery
can't survive it.
1 (SRV_FORCE_IGNORE_CORRUPT)
Let the server run even if it detects a corrupt page. Try to make SELECT * FROM tbl_name
jump over corrupt index records and pages, which helps in dumping tables.
http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
Again, you should see chatter from InnoDB in the MySQL error log.
Final suggestion, don't use workbench. Use the mysql
command line client to explore the server schemata and use mysqldump
directly from the command line to try to restore individual schemas or individual databases.
update/additional:
how do you recover stored procedures? I get an error because they access corrupt tables
It shouldn't be because the procedures access corrupt tables, because stored procedures and functions aren't validated against their referenced tables in the way views are (drop a table that a view references, and then SHOW CREATE VIEW
doesn't work -- the same isn't the case with procs or functions)... MySQL doesn't care whether tables referenced by stored procedures and functions even exist (except at runtime)... however, unlike view definitions, which are stored in individual files, the definitions of procedures and functions are stored in the proc
table in the mysql
schema, which is a MyISAM table... so I would expect that you'd see error messages referencing that table and CHECK TABLE mysql.proc;
and REPAIR TABLE mysql.proc;
might get you going again. Note that the version of MySQL Server that you're using for your recovery needs to be the same release series (e.g. 5.5) or the table definition of mysql.proc
will be wrong... that shouldn't be the issue here, since if I read the question correctly, the crashed server and the recovery server are both 5.5.31.
If you can SELECT * FROM mysql.proc
then you can retrieve the definitions that way if for some reason the SHOW CREATE PROCEDURE
statement doesn't work.
And what options for a (partial) rescue are there for the tables that yielded an error in the dumps?
It depends on the nature and scope of the errors encountered. One option, discussed here, involves repeated INSERT ... SELECT ... LIMIT ... OFFSET
, extracting blocks of rows from readable pages into a MyISAM table (because you can't write to an InnoDB table while innodb_force_recovery
is on). There's also the Percona Data Recovery Tool for InnoDB, though I have never had occasion to use it.
Everything you have done is correct in order to drastically defragment innodb tablespaces (there are other ways, but exporting, deleting files and importing certainly works).
Trying to optimize the tables (which, by the way, the correct way to do it for innodb is running ALTER TABLE mytable ENGINE=InnoDB
- OPTIMIZE table just calls this, and REPAIR table does nothing for InnoDB) just after an in-primary-key-order import is useless. The way InnoDB works is optimising for access and write performance, not for data size. You will always have overhead in disk space (even if free space says 0), as innodb reserves spaces in whole extend beyond a certain size. Also, some data types, like blobs and random inserts and deletions can lead to extra fragmentation, but I presume that the size you are showing is less than 10% of the table size, so you should no worry. After a proper import, it is the least fragmented way innodb can work with the default options. Under normal operation, the file size should be constant when inserting new data until no more free space is available.
Of course, you can try to change parameters like extent or page sizes, but if you are concerned about saving disk space over performance I would recommend you to try compression (InnoDB Barracuda file formant has it) or use a different engine. However, please note that fragmentation and having huge files with lots of free space is usually not a concern with innodb_file_per_table = 1
under normal loads.
Best Answer
I found a cause. All folders under the data directory were hidden attribute. I changed them then I can see tables and dump data.