Is it safe to assume columns of NVARCHAR, NTEXT, NCHAR, BIT, INT, DECIMAL, FLOAT, and DATETIME all MUST be UNICODE...
Only the XML
and N
-prefixed types (NCHAR
, NVARCHAR
, and NTEXT
[which has been deprecated since SQL Server 2005 was released so please do not use it]) are Unicode. Those other types you mentioned are not strings and are not stored as strings, hence they are not relevant to this question.
... and therefore WILL NOT have any characters unable to be converted from UTF-16LE to UTF-8...
This is not exactly a valid question. Unicode characters are Unicode characters regardless of their encoding, whether it is UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE. Now, it is possible for the data itself to contain invalid sequences, such as invalid Surrogate Pairs. But then those aren't valid characters in the UTF-8 or UTF-32 encodings either.
... simply by exporting the values to a .txt file and resaving them with the UTF-8 Encoding prior to importing them to MySQL?
Well, you need to be sure to save the initial export file with a Unicode-encoding. So you would use either the -N
or -w
options with BCP.
Also, make sure that you are doing more than just changing the Byte Order Mark (BOM) of the file and are actually converting the Unicode / UT-16LE characters to UTF-8.
I ran into the same problem when trying to use SQL Migration Wizard to create a local copy of my database. I know my source data didn't have duplicate key values, so I came up with the following script. It simply causes duplicate key entries to be ignored (and a warning is generated instead of error). Please read the comments in the code block below for instructions.
-- For some reason, SQL Azure Migration Wizard tries to insert duplicate records and fails.
-- I've wasted a lot of time making many many many attempts at copying the database, failing repeatedly,
-- So this is the workaround:
-- 1.0 In SQL Azure Migration Wizard: After selecting the database to script, select the "Advanced" button.
-- 1.1 Change the "Script Table / Data" property from "Table Schema with Data" to "Table Schema Only".
-- 1.2 Click "Next" and proceed until the "Schema Only" migrate script has been executed a new database.
-- 2. Run THIS Sql script with @IGNORE_DUP_KEY set to 'ON' against the TARGET database.
-- 3.0 Back in SQL Azure Migration Wizard, click "Back" until you see the advanced button again, and click it.
-- 3.1 Change the "Script Table / Data" property from "Table Schema with Data" to "Data Only".
-- 3.2 Click "Next" and proceed until the script has been executed. NOTE: Table Name may still appear red like it failed, but looking at the message will show you that it was just a warning.
-- TO VERIFY: "SELECT COUNT(*) FROM TableName" on any tables that had the warning show in the source and target databases, and compare the counts.
-- 4. Run THIS Sql script, but set @IGNORE_DUP_KEY to 'OFF' to revert back to original/expected behavior.
DECLARE @IGNORE_DUP_KEY VARCHAR(MAX) = 'ON'; -- Options: ON, OFF
DECLARE @TableName VARCHAR(255)
DECLARE @sql NVARCHAR(500)
DECLARE TableCursor CURSOR FOR
SELECT OBJECT_SCHEMA_NAME([object_id])+'.'+name AS TableName
FROM sys.tables
OPEN TableCursor
FETCH NEXT FROM TableCursor INTO @TableName
WHILE @@FETCH_STATUS = 0
BEGIN
SET @sql = 'ALTER TABLE ' + @TableName + ' REBUILD WITH (IGNORE_DUP_KEY = ' + @IGNORE_DUP_KEY + ');'
--print @sql
EXEC (@sql)
FETCH NEXT FROM TableCursor INTO @TableName
END
CLOSE TableCursor
DEALLOCATE TableCursor
GO
I know this solution is being posted some time after the original question was asked, but hopefully this will help someone else out!
Best Answer
If you want to quickly migrate your data from MS SQL Server to MySQL, I suggest using the Migration tool from Oracle. This can really save you some time and effort.
This article can help you get started:
http://www.infoq.com/news/2012/08/MySQL-Migration