Sql-server – supplementary characters in FreeTDS

freetdssql serversql-server-2008unicodeunixodbc

I have a C++ application that connects to SQL Server 2008 through FreeTDS unixODBC on an RHEL box.

However, i am unable to insert supplementary characters to the database through FreeTDS unixODBC.

[FreeTDS][SQL Server]Error converting characters into server's character set. Some character(s) could not be converted

This is the query that my application is hitting

INSERT INTO mytable (SAMPLE) VALUES(N'乕乭??乺丕')

== version details ==
unixODBC version - 2.3.0
freeTDS version - 0.95

I can successfully insert the characters in BMP.
This characters are inserted perfectly fine.
INSERT INTO mytable (SAMPLE) VALUES(N'乕乭乺丕')

While using Microsoft SQL Server Native Client for Linux, the supplementary characters are inserted properly.

== freetds conf ==
[mysever.mydomain.com]
host = mysever.mydomain.com
port = 1433
tds version = 7.3
debug flags = 0xffff
dump file = /tmp/dump.log
dump file append = no
client charset = UTF-16
use utf-16 = yes

unixODBC v2.3.0 has been built with

./configure --prefix=/usr/local/ --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE

FreeTDS v0.95 has been built with

./configure --prefix=/usr/local --sysconfdir=/usr/local/etc --with-tdsver=7.3

Best Answer

Things to look at / try:

  1. Looking at the documentation again (now that you have posted the config file and configuration options), I think your setting of client charset is incorrect. You specify --with-iconv-char-enc=UTF-8 when configuring the build for the ODBC driver, so I suspect you need to also use UTF-8 when specifying the client charset option (or whatever encoding your app code is using).

  2. Try rebuilding FreeTDS with --enable-odbc-wide-tests. This might help narrow down where the issue is (assuming the test uses Supplementary Characters).

  3. Did you see the note about "Version 0.95 removed support for iconv" on the FreeTDS page for How to build: Configure and make? Not sure what, if any, impact this has, but if nothing else helps then you need to verify this part (i.e. that your build of iconv works and that it is being used).

  4. Looking at the unixODBC website, the main page has a version history and you claim to be using 2.3.0 which, according to that history, was released in April 20th, 2010 (almost 6 years ago). Since then there have been several "wide character"-related bug fixes, as well as many other fixes. Even if some other change (#1 perhaps) fixes the immediate issue, I would still upgrade to the most recent version of this driver, 2.3.4, which was released on August 31st, 2015.

  5. Something to consider, perhaps, but if the Microsoft driver does work, why not just switch to using that?

Related Question