Mysql – Why MySQL Database is Bigger After Dump/Import

backupimportmigrationMySQLmysqldump

I have 2 VPS and plan to migrate the DB from first to second. The original DB is taking around 15GB of storage and I manage to dump out a 7GB SQL. However, when I imported the SQL to second VPS, it takes up 2x of storage becoming 32GB.

My questions are:

What might cause my DB from 15GB to become 32GB by just dump+import?
How can I avoid this?

Below are some further detail:

Both VPS are not exactly same and so do the MySQL version. The source DB is version 5.5.47. The receiving DB is version 5.7.13.
The receiving DB is newly install and empty.
The command I used to dump out SQL is: mysqldump -u <username> -p --all-databases --add-drop-database --max_allowed_packet=16777216 --single-transaction > full_dump.sql. The reason I am setting max_allowed_packet is to have the same value compare to receiving DB
Not sure if this is relevant. If I do not specify single-transaction in my mysqldump command, it will error mysqldump: Got error: 2006: MySQL server has gone away when using LOCK TABLES
The source DB storage distribution is different from the receiving DB:

14858M /var/lib/mysql

14483M /var/lib/mysql/ibdata1

365M /var/lib/mysql/wordpress
This is the receiving DB storage distribution:

31558M /var/lib/mysql

31306M /var/lib/mysql/wordpress

337M /var/lib/mysql/wordpress/wp_wfLeechers.ibd

101M /var/lib/mysql/wordpress/wp_1059_posts.ibd
I've tried to run table optimisation, but no impact

Best Answer

The first thing that caught my eye was the versions of MySQL you are using.

You said

The source DB is version 5.5.47
The receiving DB is version 5.7.13

These two versions of MySQL use different default values for innodb_file_format

MySQL 5.7 has new ways of storing InnoDB as mentioned in the Documentation. Please note:

The Barracuda file format is required to use Compressed or Dynamic row formats and associated features such as compression, off-page storage for large variable-length columns, and large index key prefixes (see innodb_large_prefix). This restriction does not apply to tables stored in general tablespaces.

If your data has large variable-length columns, the result of reloading could be the storage of data and index info outside of your BTree Indexes and extra splitting of pages. See DYNAMIC and COMPRESSED Row Formats for more details on this. Many of the new InnoDB file format features are deprecated in MySQL 5.7 and will eventually disappear in future releases.

SUGGESTION #1

Even though deprecated, you could set innodb_file_format to Antelope on the MySQL 5.7 server, restart MySQL, and reload the mysqldump.

SUGGESTION #2

Use MySQL 5.5/5.6 as the Master instead of MySQL 5.7. Restart MySQL on the Master, and reload mysqldump into the Master.

SUGGESTION #3

Run ALTER TABLE innodbtable ROW_FORMAT=COMPRESSED; on all the MySQL 5.7 InnoDB tables.

GIVE IT A TRY !!!

ASPECT #1

The first thing that caught my eye was this line

InnoDB: Error: trying to load index PRIMARY for table /

This indicates you have a table using the InnoDB Storage Engine

What is interesting about InnoDB is the way a PRIMARY KEY is stored. It is stored in a structure called the gen_clust_index, or more commonly known as the Clustered Index.

My immediate guess is that a certain PRIMARY KEY entry is too big

Please consider some articles on the good, the bad, and the ugly of using long PRIMARY KEYs:

then see if the <DB Hidden>.<Table Hidden> needs to be redesigned.

ASPECT #2

In terms of your conjecture concerning a parallel truncate table, that sounds kind of dangerous. Why? InnoDB performs TRUNCATE TABLE as DDL not DML. I have written about this before:

Jul 09, 2012 : What can cause TRUNCATE TABLE to take a really long time?
Jan 17, 2012 : Problem with InnoDB "per table" file sizes
Sep 28, 2011 : How to Recover an InnoDB table whose files were moved around

ASPECT #3

Some tuning suggestions

Please add the following to my.ini

[mysqld]
max_allowed_packet=1G
innodb_fast_shutdown=0

Start mysql

In another session, run tail -f <errorlogfile> and watch InnoDB Crash Recovery.

If mysql is fully started back up and InnoDB crash recovery has completed, try to shut mysql down immediately. You may need to resize your InnoDB Transaction Logs.

Sorry for these wild suggestions, but I am flying blind here.

Please post the following in the question:

your entire my.cnf
how much RAM is on board

UPDATE 2012-12-05 12:09 EDT

Please do the following:

STEP 01) Add these changes to my.cnf

[mysqld]
max_allowed_packet=1G
innodb_fast_shutdown=0
innodb_thread_concurrency=0

STEP 02) service mysql restart

to make sure mysql comes up

STEP 03) You need to resize ib_logfile0 and ib_logfile1 (24M might be too small)

service mysql stop
cd /var/lib/mysql
mv ib_logfile0 ib_logfile0.bak
mv ib_logfile1 ib_logfile1.bak

STEP 04) Add these changes to my.cnf

[mysqld]
innodb_log_file_size=512M
innodb_log_buffer_size=8M

STEP 05) service mysql start

mysqld will recreate ib_logfile0 and ib_logfile1 512M each

Now, try and see what happens....

UPDATE 2012-12-05 12:18 EDT

In the meantime, please read my ServerFault post on the mysql packet and its sizing implication with regard to the innodb_log_file_size and innodb_log_buffer_size as I learned from someone else's ServerFault post.

UPDATE 2012-12-05 14:28 EDT

I edited all references to customer tables out of this question.

The root cause was a damaged page in ibdata1 with data and index pages mixed inside. I helped Andrew migrate data out, recreate ibdata1 with innodb_file_per_table, and Andrew reloaded the data.