Mysql show no more than 5 concurrent connection

load-testingMySQLPHP

I'm performing some load tests on my brand new server Apache/PHP/Mysql (Bitnami LAMP stack ). I'm using AWS with 1 EC2 instance behind a loadBalancer. At the moment I'm using loader.io for the tests. I'm testing a very simple API that perform this query:

select *,sleep(0.5) from debug limit 1

I added the sleep(0.5) because I wanted to see how the server behave with multiple concurrent connections and I found a bottleneck: if I run "SHOW PROCESSLIST" I can see only 5 process even if I have 10 concurrent user. The load test show that the connections are queued because the response time is growing during the test from 500 milliseconds to several seconds (depending on the duration of the test and the number of concurrent users).

I checked

select @@max_connections

and it's 151 (the default). max_user_connections is 0. What other parameter should I check to increase the number of concurrent connection on my DB?

If I run the test with 5 concurrent users, each one get a response in 500 milliseconds. If I add more concurrent users than the response time slow down.

If I run the load test on an API that does not access the DB there are no issues even with 400 concurrent users.

Monitoring with HTOP I see:

Tasks: 34, 245 thr; 2 running

Could be here the issue?

Thanks a lot

Best Answer

The solution was actually very easy:

The Bitnami LAMP stack I'm using is configured with PHP-FPM, so the configuration is not on the apache side but in php/etc/common.conf

pm=ondemand
pm.max_children=5
pm.start_servers=2
pm.min_spare_servers=1
pm.max_spare_servers=3

inreasing the pm.max_children=5 solved the issue.

Related Solutions

Mysql – Possible to make MySQL use more than one core

I actually discussed innodb_thread_concurrency with a MySQL Expert at the Percona Live NYC conference back in May 2011.

I learned something surprising: In spite of the documentation, it is best to leave innodb_thread_concurrency at 0 (infinite concurrency). That way, InnoDB decides the best number of innodb_concurrency_tickets to open for a given MySQL instance setup.

Once you set innodb_thread_concurrency to 0, you can set innodb_read_io_threads and innodb_write_io_threads (both since MySQL 5.1.38) to the maximum value of 64. This should engage more cores.

MySQL connection count much larger than query count

Here is something interesting to remember : the status variables Questions takes a count of all queries executed by mysqld, regardless of who issued each query.

Why do I say regardless of who issued each query ?

There are queries that are launched internally by mysqld. Here is a simple example:

mysql> show global status like 'Com_select'; show global status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Com_select    | 15    |
+---------------+-------+
1 row in set (0.00 sec)

+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 56    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> show global status like 'Com_select'; show global status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Com_select    | 15    |
+---------------+-------+
1 row in set (0.00 sec)

+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 58    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> show global status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 59    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> show global status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 60    |
+---------------+-------+
1 row in set (0.00 sec)

mysql>

Please note that issuing a simple query like show global status like 'Questions'; is a query even if your are just requesting internal data from mysqld.

Multiply this by any number of DB Connections coming from MONyog, Munin, Nagios, MySQL Enterprise Monitor, or any other MySQL Monitoring tools and will always have a steady rise in the number of lightweight queries being executed.

I wrote about this before in ServerFault in a question "1 billion mysql queries in 24 hours? Can something be wrong?"

Queries can quitely come and go. The question left to answer is, why so many DB Connections? You need to make sure Apache is closing DB Connections at the same rate as mysqld. You may need to see run netstat and look for DB Connections to mysql that have the TIME_WAIT status.

There are bug reports on this going back to MySQL 4.1 although the root of the problem may actually be with PHP:

Best Answer

Related Solutions

Mysql – Possible to make MySQL use more than one core

MySQL connection count much larger than query count

Related Question