After searching for a solution for the last 6 hours, I have come up dry in my attempt to add SSL to the replication. I managed to get it to connect with SSL via the mysql command line tool without issues, however I cannot seem to solve this replication issue. Based on the research I did find, this is an extremely generic catch-all SSL error.
System 1:
OS: Fedora 30 Modular
Kernel: 5.0.16-300
Arch: x86_64
MariaDB Server: 10.3.16
OpenSSL: 1.1.1c FIPS
MariaDB [(none)]> STATUS;
--------------
mysql Ver 15.1 Distrib 10.3.16-MariaDB, for Linux (x86_64) using readline 5.1
Connection id: 42
Current database:
Current user: root@localhost
SSL: Cipher in use is TLS_AES_256_GCM_SHA384
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server: MariaDB
Server version: 10.3.16-MariaDB-log MariaDB Server
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
UNIX socket: /var/lib/mysql/mysql.sock
Uptime: 18 min 0 sec
Threads: 11 Questions: 32 Slow queries: 0 Opens: 17 Flush tables: 1 Open tables: 11 Queries per second avg: 0.029
--------------
MariaDB [(none)]> SHOW SLAVE STATUS \G;
*************************** 1. row ***************************
Slave_IO_State: Connecting to master
Master_Host: REDACTED
Master_User: REDACTED
Master_Port: REDACTED
Connect_Retry: 60
Master_Log_File: master1-bin.000012
Read_Master_Log_Pos: 364174
Relay_Log_File: master1-relay-bin.000001
Relay_Log_Pos: 4
Relay_Master_Log_File: master1-bin.000012
Slave_IO_Running: Connecting
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 364174
Relay_Log_Space: 256
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: Yes
Master_SSL_CA_File: /etc/pki/tls/certs/mariadb-chain.pem
Master_SSL_CA_Path: /etc/pki/tls/certs/
Master_SSL_Cert: /etc/pki/tls/certs/mariadb.pem
Master_SSL_Cipher: TLS_AES_256_GCM_SHA384
Master_SSL_Key: /etc/pki/tls/private/mariadb.pem
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: Yes
Last_IO_Errno: 2026
Last_IO_Error: error connecting to master 'REDACTED@REDACTED:REDACTED' - retry-time: 60 maximum-retries: 86400 message: SSL connection error: error:00000000:lib(0):func(0):reason(0)
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 0
Master_SSL_Crl: /etc/pki/tls/certs/mariadb-chain.pem
Master_SSL_Crlpath: /etc/pki/tls/certs/
Using_Gtid: No
Gtid_IO_Pos:
Replicate_Do_Domain_Ids:
Replicate_Ignore_Domain_Ids:
Parallel_Mode: conservative
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Slave_DDL_Groups: 0
Slave_Non_Transactional_Groups: 0
Slave_Transactional_Groups: 0
1 row in set (0.000 sec)
ERROR: No query specified
MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%ssl%';
+---------------------+-------------------------------------------+
| Variable_name | Value |
+---------------------+-------------------------------------------+
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | /etc/pki/tls/certs/mariadb-chain-x509.pem |
| ssl_capath | |
| ssl_cert | /etc/pki/tls/certs/mariadb-x509.pem |
| ssl_cipher | TLS_AES_256_GCM_SHA384 |
| ssl_crl | |
| ssl_crlpath | |
| ssl_key | /etc/pki/tls/private/mariadb.pem |
| version_ssl_library | OpenSSL 1.1.1c FIPS 28 May 2019 |
+---------------------+-------------------------------------------+
10 rows in set (0.002 sec)
System 2:
OS: Fedora 30 Modular
Kernel: 5.0.16-300
Arch: x86_64
MariaDB Server: 10.3.16
OpenSSL: 1.1.1c FIPS
MariaDB [(none)]> STATUS;
--------------
mysql Ver 15.1 Distrib 10.3.16-MariaDB, for Linux (x86_64) using readline 5.1
Connection id: 60
Current database:
Current user: root@localhost
SSL: Cipher in use is TLS_AES_256_GCM_SHA384
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server: MariaDB
Server version: 10.3.16-MariaDB-log MariaDB Server
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
UNIX socket: /var/lib/mysql/mysql.sock
Uptime: 40 min 44 sec
Threads: 12 Questions: 623 Slow queries: 0 Opens: 48 Flush tables: 1 Open tables: 42 Queries per second avg: 0.254
--------------
MariaDB [(none)]> SHOW SLAVE STATUS \G;
*************************** 1. row ***************************
Slave_IO_State: Connecting to master
Master_Host: REDACTED
Master_User: REDACTED
Master_Port: REDACTED
Connect_Retry: 60
Master_Log_File: master1-bin.000007
Read_Master_Log_Pos: 344
Relay_Log_File: master1-relay-bin.000006
Relay_Log_Pos: 4
Relay_Master_Log_File: master1-bin.000007
Slave_IO_Running: Connecting
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 344
Relay_Log_Space: 256
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: Yes
Master_SSL_CA_File: /etc/pki/tls/certs/mariadb-chain.pem
Master_SSL_CA_Path:
Master_SSL_Cert: /etc/pki/tls/certs/mariadb.pem
Master_SSL_Cipher:
Master_SSL_Key: /etc/pki/tls/private/mariadb.pem
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: Yes
Last_IO_Errno: 2026
Last_IO_Error: error connecting to master 'REDACTED@REDACTED:REDACTED' - retry-time: 60 maximum-retries: 86400 message: SSL connection error: error:00000000:lib(0):func(0):reason(0)
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 0
Master_SSL_Crl: /etc/pki/tls/certs/mariadb-chain.pem
Master_SSL_Crlpath:
Using_Gtid: No
Gtid_IO_Pos:
Replicate_Do_Domain_Ids:
Replicate_Ignore_Domain_Ids:
Parallel_Mode: conservative
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Slave_DDL_Groups: 0
Slave_Non_Transactional_Groups: 0
Slave_Transactional_Groups: 0
1 row in set (0.000 sec)
ERROR: No query specified
MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%ssl%';
+---------------------+--------------------------------------+
| Variable_name | Value |
+---------------------+--------------------------------------+
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | /etc/pki/tls/certs/mariadb-chain.pem |
| ssl_capath | |
| ssl_cert | /etc/pki/tls/certs/mariadb.pem |
| ssl_cipher | |
| ssl_crl | |
| ssl_crlpath | |
| ssl_key | /etc/pki/tls/private/mariadb.pem |
| version_ssl_library | OpenSSL 1.1.1c FIPS 28 May 2019 |
+---------------------+--------------------------------------+
10 rows in set (0.005 sec)
I'm trying to setup both servers as master and slave for full replication. It was working until I went to implement the SSL. I'm trying to use Let's Encrypt certificates. I have already converted the private key to RSA and made a full copy of the certificate and chain, so it's not just a symlink. Both servers are running on the same port (non-standard) and have the same users and passwords. I have completely disabled SELinux, to no avail.
the permissions should be fine…
ls -l /etc/pki/tls/*/mariadb*.pem
-rw-r--r--+ 1 mysql mysql 3566 Aug 11 02:17 /etc/pki/tls/certs/mariadb-chain.pem
-rw-r--r--+ 1 mysql mysql 1919 Aug 11 02:17 /etc/pki/tls/certs/mariadb.pem
-rw-r--r--+ 1 mysql mysql 1679 Aug 11 02:17 /etc/pki/tls/private/mariadb.pem
Thanks for your time.
UPDATE:
I tried changing the permissions on the PEM files to 600, but it did not fix it. I managed to get it logging at maximum verbosity and this is the section pertinent to the error:
2019-08-14 16:42:53 10 [ERROR] Slave I/O: error connecting to master 'REDACTED@REDACTED:REDACTED' - retry-time: 60 maximum-retries: 86400 message: SSL connection error: error:00000000:lib(0):func(0):reason(0), Internal MariaDB error code: 2026
2019-08-14 16:43:54 12 [Warning] IP address 'REDACTED' could not be resolved: Name or service not known
2019-08-14 16:43:54 12 [Warning] Aborted connection 12 to db: 'unconnected' user: 'unauthenticated' host: 'REDACTED' (CLOSE_CONNECTION)
I also removed the ssl_cipher option from the server I forgot to remove it from, so the cipher configs match.
Best Answer
I had that same error when replicating from a
mysql 5.6.44to amariadb 10.4.For me it was caused by
mysqlonly supportingTLSv1andmariadbrequiringTLSv1.1.My solution was to update
mysqlto a version5.6.46(or higher) because it supportsTLSv1.1starting from5.6.46.