Mysql – Are newly inserted data automatically stored in InnoDB buffer pool

innodbMySQL

I understand that innoDB will store select statement result in the buffer pool, does InnoDB store newly inserted data (insert into statement) into buffer pool even after data has committed and wrote to disk?

Best Answer

Yes and no. Yes, buffer pool will contain the recently inserted data. No, the buffer pool doesn't contain SELECT statement output, that is what the query cache does, but it is generally harmful, deprecated and removes in MySQL 8.0. The buffer pool contains InnoDB pages (both index and data).

Related Solutions

Mysql – How much memory do I need for innodb buffer pool

If you go strictly by that rule of accommodating an addition 10%, here is my suggestion:

SELECT CONCAT(CEILING(RIBPS/POWER(1024,pw)),SUBSTR(' KMGT',pw+1,1))
Recommended_InnoDB_Buffer_Pool_Size FROM
(
    SELECT RIBPS,FLOOR(LOG(RIBPS)/LOG(1024)) pw
    FROM
    (
        SELECT SUM(data_length+index_length)*1.1*growth RIBPS
        FROM information_schema.tables AAA,
        (SELECT 1 growth) BBB
        WHERE ENGINE='InnoDB'
    ) AA
) A;

This will produce exactly what you need to set innodb_buffer_pool_size in /etc/my.cnf. If you want to account for 25% increase in data and indexes over time, please change (SELECT 1 growth) BBB to (SELECT 1.25 growth) BBB

Recently, I answered another question like this in the DBA StackExchange.

MySQL buffer pool usage + high load

You have a read/write ratio of 26:1 (based on 10869877 divided by 406010), which is 96% reads.

I think you might be suffering from what I call READ AHEAD FUTILITY. Sounds like I made that up. Well, you are right. I did. What do I mean ? According to the MySQL Documentation on the status variable Innodb_buffer_pool_read_ahead_evicted

The number of pages read into the InnoDB buffer pool by the read-ahead background thread that were subsequently evicted without having been accessed by queries.

Tons of SELECTs may read too many pages of data for the purposes of doing read ahead. Those pages may never be accessed and are simply evicted from the Buffer Pool. It's like blowing up a gigantic balloon without tying a knot. The air will eventually come out. This is true even for a full Buffer Pool. Pages are not allowed to lie dormant. They must be removed to make room for other moving parts of the InnoDB Architecture

InnoDB Architecture

For example, look at the diagram. Were you aware that up to 25% of the Buffer Pool is used as a scratch pad for non-unique index changes? It forms a conduit that migrates those changes into the system tablespace. This can crash InnoDB if there are heavy INSERTs and UPDATEs into tables that have many nonunique indexes. You would drop those indexes, do your INSERTs, and create the indexes again to circumvent this. I wrote about this before

Jul 30, 2014 : InnoDB import performance
May 12, 2014 : Buffer pool above 90% utilized causes MySQL to crash

Looking back at your chart, your miscellaneous is non existent. What does that indicate?

According to MySQL Documentation on Innodb_buffer_pool_pages_misc

The number of pages that are busy because they have been allocated for administrative overhead such as row locks or the adaptive hash index. This value can also be calculated as Innodb_buffer_pool_pages_total – Innodb_buffer_pool_pages_free – Innodb_buffer_pool_pages_data.

Apparently, you are not doing frequent searches for the same data. Your SELECTs must be doing

large index range scans
lots of index range scans
lots of table scan
I discussed this in MySQL status variable Handler_read_rnd_next is growing a lot

SUGGESTIONS

You probably need to tune your queries so they do not request so much data.
You should also avoid doing mysqldumps in the middle of the business day and this tends to load every data page and its grandmother into the Buffer Pool, only to have those pages quietly evicted. If you have to run mysqldumps, setup a MySQL Slave and o your dump from the Slave.
You could experiment with disabling the change buffering (set innodb_change_buffering to 0).

UPDATE 2014-08-11 16:45 EDT

Please look back at the comment section. Derek Downey gave you a link to my 3 year old post. Based on the 2012 update in my post, you need to set innodb_buffer_pool_instances = 2. This will help curb mysqld's swap behavior.

From your SHOW ENGINE INNODB STATUS\G, you have too few read and write threads

--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)

You only have 4 read and 4 write I/O threads. Bump up your threads

innodb_read_io_threads = 16
innodb_write_io_threads = 16

If you had more that 2 cores, I would use 32 or 64. You can try 16 for now.

InnoDB is always at odds with the Query Cache. Please disable it.

query_cache_size = 0

You need a bigger log buffer for better InnoDB write performance

innodb_log_buffer_size = 256M

These are all the changes

[mysqld]
innodb_buffer_pool_instances = 2
innodb_log_buffer_size = 256M
innodb_read_io_threads = 16
innodb_write_io_threads = 16
query_cache_size = 0