Mysql – bulk_insert_buffer_size and InnoDB

innodbmyisamMySQL

I read the following on this page

http://dev.mysql.com/doc/mysql/en/server-system-variables.html

bulk_insert_buffer_size MyISAM uses a special tree-like cache to make
bulk inserts faster for INSERT … SELECT, INSERT … VALUES (…),
(…), …, and LOAD DATA INFILE. This variable limits the size of the
cache tree in bytes per thread. Setting it to 0 disables this
optimization. Note: This cache is used only when adding data to a
non-empty table. The default value is 8MB. This variable was added in
MySQL 4.0.3. This variable previously was named
myisam_bulk_insert_tree_size.

I take this to mean that bulk_insert_buffer_size has no bearing on
InnoDB tables, only on MyISAM tables. Is this correct?

Best Answer

Yes, you are correct. I wrote about this before: How can I optimize this mySQL table that will need to hold 5+ million rows?

WHen it comes to bulk loading InnoDB tables, one of the most frequently ignored settings to disable is innodb_doublewrite.

InnoDB Architecture

InnoDB Architecture

Please note the double write buffer inside the system tablespace file ibdata1. Every time a 16K page is written to an InnoDB table or transaction log, the page is written to the double wrote buffer first. All you need to do is disable it, load your data, and reenable it.

STEP 01 : Disable Fast Shutdown

mysql -uroot -p... -ANe"SET GLOBAL innodb_fast_shutdown = 0"

STEP 02 : Restart mysql without the double write buffer

service mysql restart --innodb-doublewrite=0

STEP 03 : Load the InnoDB data

STEP 04 : Restart mysql

service mysql restart

CAVEAT

You can disable the foreign key checks and the unique checks as well. Any standard mysqldump will do that for you already.