Mysql – How to determine that the maximum size of the database file is exceeded, and switch the data insertion to the second server

Performance shouldn't be a concern: security, consistency, data integrity should be

• Security: If each web site has a database this implies that the data for that web site is private and separate. So 40000 databases with 10 tables each.

• Consistency: managing 40k websites requires utter consistency. Why pick 400x1000 or 4000x100? How will you separate tables? How do you know which site uses which set of tables in which DB?

• Data integrity: If you restore one database, do you really want to reset the data for another 399 or 3999 websites too?

So: 40k databases, one per web site.

Finally, do you have 40k web sites or is this premature optimisation?

First you need to do is run this query:

SELECT user,host FROM mysql.user
WHERE super_priv='Y' AND
CONCAT(user,'@',host) <> 'root@localhost';

This will list all users that have SUPER privilege. Most users that do application-related DB processing do not require this privilege. According to the MySQL Documentation, those with SUPER privilege can do the following:

• Run CHANGE MASTER TO for controlling replication coordinates
• KILL or mysqladmin kill to kill threads belonging to other accounts
• PURGE BINARY LOGS to systemically delete binary logs
• Make configuration changes using SET GLOBAL to modify global system variables
• mysqladmin debug command
• enabling or disabling logging
• performing updates even if the *read_only* system variable is enabled
• starting and stopping replication on slave servers
• specification of any account in the DEFINER attribute of stored programs and views
• HERE IS THE MOST IMPORTANT ONE FOR YOUR PROBLEM: : Enables you to connect (once) even if the connection limit controlled by the max_connections system variable is reached.

You will need to login as root@localhost and revoke SUPER privilege as follows:

UPDATE mysql.user SET super_priv='N'
WHERE super_priv='Y' AND
CONCAT(user,'@',host) <> 'root@localhost';
FLUSH PRIVILEGES;

Once you do this, whenever all users flood mysql connections, only root@localhost can login. After all, if everybody and his grandmother had SUPER privilege, this would bar root@localhost from ever connecting ahead of everybody else. If max_connections is at 200 and you need to raise it to 300 without having to restart mysqld, you can dynamically increase the max_connections with this command:

mysql> SET GLOBAL max_connections = 300;

That will allow more connections effective immediately, but don't just arbitrarily increase the number on whim. You have to make sure mysql has enough RAM to accommodate the increase.

CAVEAT : If you change max_connections dynamically to 300, please put it in /etc/my.cnf

[mysqld]
max_connections=300

You can run mysqltuner.pl on your MySQL DB Server. If you do not have it, then run the following:

cd
wget mysqltuner.pl
perl mysqltuner.pl

The 3rd line under Performance Metrics has this

-------- Performance Metrics -------------------------------------------------
[--] Up for: 8d 20h 46m 22s (8M q [10.711 qps], 129K conn, TX: 90B, RX: 19B)
[--] Reads / Writes: 4% / 96%
[--] Total buffers: 2.1G global + 5.4M per thread (2000 max threads)
[OK] Maximum possible memory usage: 12.6G (80% of installed RAM)

See the 5.4M per thread? That is multipled by max_connections. In this example, that would be a maximum of about 10.8G of RAM. Therefore, each time you bump up max_connections, you should run mysqltuner.pl and check if you are pressing the OS for too much memory.

In any case, limiting who has SUPER privileges give such users opportunity to mitigate flooding mysqld with DB Connections.