Mysql – Getting ERROR 1045 (28000): Access denied for user even with proper setup and on different connections

MySQL

I am getting this error: ERROR 1045 (28000): Access denied for user 'ipcuser'@'10.0.0.26' (using password: YES)

The database is Percona MySQL 5.6 and is on the 10.0.0.201 IP

We have servers in the 10.0.0.x range. Testing from different servers we get different results using the exact same command: mysql -u ipcuser -pPASSHERE -h 10.0.0.201 -D ipc --protocol=tcp
The issue came up over night when nothing has changed and no one was working. No code changes at all. No database changes. The database was last restarted 4 days ago before attempting today.

The results of trying to connect from: 

10.0.0.200 Connects  
10.0.0.70 Connects  
10.0.0.26 Access Denied  
10.0.0.20 Access Denied  
10.0.0.101 Access Denied

Output from: SELECT user, host FROM mysql.user; 

ipcuser     10.0.0.%  
ipcuser     10.0.0.101  
ipcuser     localhost  
ipcuser     127.0.0.1  
root        10.0.0.201  
root        127.0.0.1  
root        ::1  
root        localhost

Already tested:
– Restarting MySQL
– Adding in the specific IP of the server (that would be the 10.0.0.101 address in the above output, removing it has no change)
– Restarting the network on both the DB and failed connections
– Checking the error log. Nothing.
– Disabling firewalls. None of the connecting servers nor the DB had iptables running.
– Running FLUSH HOSTS
– Changing the bind

Also for some extra weirdness, I was able to connect once from the 10.0.0.101 IP, but after issuing a query it disconnected me and on attempting to reconnect gave access denied.

After more testing, I shut the MySQL server down and the servers failed to connect were still getting a specific access denied error. The ones that had successful connections got could not connect.

Best Answer

The problem has been fixed. It was not due to anything MySQL.

We have multiple DB servers, another one was assigned the same IP (10.0.0.201) and neither server picked up on it when doing an ifup. This would explain why some worked and others didn't, as well as when we got MySQL responses when it was supposedly down.

Related Question