Mysql – Cache MySQL database in memory

MySQLperformanceWordpress

I am having problems with a website with 600MB of MySQL database. The website is way too slow. I noticed the bigger the MySQL database gets, the slower it goes. When it was 5MB, the website was very fast. When it started getting bigger it started to get slower and slower and now, at 600MB, it is really slow, taking like 10 seconds to load pages.

I checked the top processes and it is nothing related to high load or anything. It is not even related to IOPS as I tested on HDD 7.2k rpm drives and it gave same problem now with testing with Intel 320 SSD drives, so I don't think it is about high queries either.

The website is using WordPress and there are like 9 plugins active. People said it might be the plugins…well maybe…but right now I just want to cache the whole database in memory and would like to get help and direction on where to start and how to do it.

I have 16GB RAM and i5-2400 4 cores @ 3.1 GHz. OS is centos 5.7

top - 07:23:57 up 9 days, 12:15, 0 users, load average: 0.09, 0.04, 0.05
Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.2%us, 1.0%sy, 0.0%ni, 90.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16367532k total, 3641628k used, 12725904k free, 612140k buffers
Swap: 1046520k total, 0k used, 1046520k free, 1538896k cached

Best Answer

If I were you, I would switch all data to InnoDB. Table locking/row locking has long been discussed by many. I would always choose InnoDB hands down. However, there is another profound reason for choosing InnoDB...CACHING.

While most people boast that MyISAM is faster for reads, most people forget that the many cache for MyISAM, which is called the key cache (set by key_buffer_size), only caches index pages from .MYI files. It never caches data pages. It has an official maximum of 4GB in 32-bit Systems. 8GB is best maximum for 64-bit.

The InnoDB Buffer Pool caches the data and index pages. Depending on the server your have, you can cache up to the entire dataset in RAM. You can tune InnoDB for up to 80% RAM and 10% for DB Conenctions, and leave 10% for the OS. This is true even for different operating systems.

I have recommended these things for Drupal customers with marvelous success. It applies to Wordpress just as well. I have provided DB support for clients with WordPress. Same improvements.

You can always configure memory for InnoDB more effectively that you can more MyISAM. There is always a way to tweek InnoDB to suit your performance needs. As your data grows, it will eventually become a requirement.

UPDATE 2011-11-21 11:44 EST

If your complete dataset is small enough, you could execute a SELECT query on every table you have right after mysql starts up.

For all tables that are InnoDB and/or MyISAM, run this query:

SELECT DISTINCT
    CONCAT('SELECT ',ndxcollist,' FROM ',
    db,'.',tb,' ORDER BY ',ndxcollist,';') SelectQueryToLoadCache
FROM (
    SELECT
        engine,table_schema db,table_name tb,index_name,
        GROUP_CONCAT(column_name ORDER BY seq_in_index) ndxcollist
    FROM (
        SELECT
            B.engine,A.table_schema,A.table_name,
            A.index_name,A.column_name,A.seq_in_index
        FROM
            information_schema.statistics A INNER JOIN
            (SELECT engine,table_schema,table_name
            FROM information_schema.tables
            WHERE engine IN ('InnoDB','MyISAM')) B
            USING (table_schema,table_name)
        WHERE
            B.table_schema NOT IN ('information_schema','mysql')
            AND A.index_type <> 'FULLTEXT'
        ORDER BY
            table_schema,table_name,index_name,seq_in_index
        ) A
    GROUP BY
        table_schema,table_name,index_name
) AA
ORDER BY
    engine DESC,db,tb
;

This will output every possible SELECT query you need to run that will summon all indexes to be referenced. Place this query in a file called /root/MakeSelectQueriesToLoad.sql. Run the script and collect the output /root/SelectQueriesToLoad.sql. Finally, run it:

mysql -u... -p... -AN < /root/MakeSelectQueriesToLoad.sql > /root/SelectQueriesToLoad.sql
mysql -u... -p... < /root/SelectQueriesToLoad.sql

This will definitely preload all index pages into the InnoDB Buffer Pool and MyISAM Key Cache. If all your data is InnoDB, make two changes:

  • replace WHERE engine IN ('InnoDB','MyISAM') with WHERE engine='InnoDB'
  • replace CONCAT('SELECT ',ndxcollist,' FROM ', with CONCAT('SELECT * FROM ',

This will also populate more data pages into the InnoDB Buffer Pool.

FINAL NOTE : Make sure the InnoDB Buffer Pool is large enough to hold all your InnoDB Data

Related Question