What you're looking for is...
ESCAPED BY '"'
In CSV, one way of including literal quotation marks inside a single field is by doubling them up.
In the example file you posted, that's exactly what's happening at line 44.
The rest of your statement looks correct. Replace the ESCAPED BY
part with what I've shown above, and you should be good.
http://dev.mysql.com/doc/refman/5.5/en/load-data.html
Occurrences of the ENCLOSED BY character preceded by the ESCAPED BY
character are interpreted as part of the current field value.
...which is exactly what you're needing.
You are absolutely right. it is not replication-safe. In fact, I wrote about how LOAD DATA INFILE replicates back on Jan 22, 2012 : MySql shell command not replicated to slave. Basically, the entire data file is stored in the binary logs, replicated to the Slave, manifested as a text file in /tmp, and the LOAD DATA INFILE is executed.
What could help is setting sync-binlog=1 before running LOAD DATA INFILE. Notwithstanding, Replication of a large CSV file is at the mercy of replication and the network.
SUGGESTION
For a Text File called mydata.csv
, you are better off doing the following:
Step 01 : cp mydata.csv to mydata.csv2
Step 02 : Make the script LoadMaster.sql
SET SQL_LOG_BIN=0;
SET bulk_insert_buffer_size = 1024 * 1024 * 256;
LOAD DATA INFILE 'mydata.csv' INTO tb1 ... ;
Step 03 : Make the script LoadSlave.sql
SET SQL_LOG_BIN=0;
SET bulk_insert_buffer_size = 1024 * 1024 * 256;
LOAD DATA INFILE 'mydata.csv2' INTO tb1 ... ;
Step 04: Load both in parallel
mysql -hIPMaster -u... -p... < LoadMaster.sql &
mysql -hIPSlave -u... -p... < LoadSlave.sql &
wait
If you have multiple slaves, you could do this:
mysql -hIPMaster -u... -p... < LoadMaster.sql &
mysql -hIPSlave1 -u... -p... < LoadSlave.sql &
mysql -hIPSlave2 -u... -p... < LoadSlave.sql &
mysql -hIPSlave3 -u... -p... < LoadSlave.sql &
....
wait
That way
- both imports are done together
- binlogs are not bloated
- CSV file can always be deleted
Best Answer
Are you looking for
LOAD DATA
LOCAL
INFILE
?http://dev.mysql.com/doc/refman/5.5/en/load-data.html
Update: The original question exhibits a significant misunderstanding about
LOAD DATA INFILE
that originally escaped my attention:The client was referred to as being "local" and the server was referred to as being "remote," which makes that statement 100% incorrect.
LOAD DATA INFILE
requires that the file already be on the server's filesystem, and addingLOCAL
means it must be on the client's filesystem.From the documentation: