Your first query is asking for 20,000 rows. what is being victimized here ???
Your Innodb Buffer Pool and Log Files.
Because of SELECT * FROM history WHERE log LIKE '%a%' ORDER BY log ASC LIMIT 0, 20000;
, all the data pages for the history table is hogging InnoDB resources.
Since you are using Amazon RDS, you cannot resize the InnoDB Log Files. They are always 128M under all RDS MySQL models. Your buffer pool may or may not be full (You have 5882511360 set for it, which is 5610M or 5.4785G). Naturally, bigger model means more IOPs.
You can run this to see how full the buffer pool is
SELECT FORMAT(A.num * 100.0 / B.num,2) BufferPoolFullPct FROM
(SELECT variable_value num FROM information_schema.global_status
WHERE variable_name = 'Innodb_buffer_pool_pages_data') A,
(SELECT variable_value num FROM information_schema.global_status
WHERE variable_name = 'Innodb_buffer_pool_pages_total') B;
Another aspect I see is the log
field (TEXT). You are asking for an ORDER BY
on this big column.
You need to change the first query to throttle the SELECT. In other words, fetch smaller chunks of history.
If you cannot change the data or the query, there is nothing you can configure except to shift to m1.xlarge (Comes with 11922309120 as the Buffer Pool Size = 11370M = 11.1035G) although log file size would still be stuck at 128M.
Infrastructure aside, it is possible for SELECTs to block INSERTs, UPDATEs, and DELETEs in InnoDB.
I wrote about this before
Back on January 21, 2009, Peter Zaitsev stated the following on mysqlperformanceblog.com
As the call for action – I would surely like someone to see if EXT3 can be fixed in this regard as it is by far the most common file system for Linux. It is also worth to investigate if something can be done on MySQL side – may be opening binlog with O_DSYNC
flag if sync_binlog=1
instead of using fsync will help ? Or may be binlog pre-allocation would be good solution.
As of yet, I know of no one having touched this issue. O_DSYNC
as a default is not an appealing prospect but does accommodate faster writes that are not really verified. That why there is so much hype around O_DIRECT
.
I can tell you do not have the InnoDB Plugin installed. With the InnoDB Plugin, several variables should exist.
You should upgrade InnoDB in one of two ways:
Once you have done so, you can enhance InnoDB to
- access more CPUs and Cores
- increase read and write I/O threads
- scale the I/O capacity (this is especially needed for different storage media)
Here are my past posts on the settings you can change for this:
Best Answer
There are two ways to collect general and slow logs:
Based on the info you provided, it looks like MySQL is using the default value for log_output which is "Log output goes to file(s) defined by general_log_file and slow_query_log_file". You can verify this by looking at these files (look in folder /usr/local/mysql/data/ and see what *.log files are there)
If you want the logs to go to database, on a MySQL client when logged in as a privileged user e.g. root:
SET GLOBAL log_output='TABLE';
SET GLOBAL general_log=ON;
SET GLOBAL slow_query_log=ON;
SET GLOBAL long_query_time=0;
The above will send global and slow logs to
mysql.general_log
andmysql.slow_log
tables.Note that I have set
long_query_time
to 0 seconds i.e. every query goes toslow_log
(that is lot of output into the table). You can change this to another value that suits your use case.Also,
general_log
produces a huge amounts of logs so you should only enable it for certain amount of time.