Mysql – Multiple Schemas for increasing MySQL (5.7) Performance

innodbMySQLmysql-5.7performanceperformance-tuning

Lets start with my question first:

Does having multiple schemas (1 main and 1 daily – having 5-6 tables with same structure as main, but only working for the day and will be synced back with the main one every night) in MySQL (5.7) server can impact its performance ?

More background:

The problem I'm facing with my database is that the application was working fine under certain load (still does but LA goes up to 8), but past few days I'm fighting over CPU usage … I dig the problem and the reason for that is that there are locks going on and my workers are stuck trying to insert/update new data.

To go even deeper I'm doing at least 1K inserts per minute, up to 2K updates per minute (some are on the previously inserted records), and more than 4-5K selects per minute.

And I'm not talking for simple integer inserts … lots of strings, timestamps, integers and 6 indexes (total column number in the table is 32)

My server is fairly powerful (huh, fairly).

  • 16GB RAM
  • 1TB SSD
  • 8 Core, Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  • Software: MySQL 5.7

Part of my my.cnf

innodb_buffer_pool_size = 12G
innodb_buffer_pool_instances = 8
innodb_log_file_size = 30G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 2
innodb_thread_concurrency = 8

and still I'm getting LA of up to 8.00

The problem with all this is because all my workers start every minute and they sort of confront each other.

What I came with is following scenario:
– create separate schema with my 'daily' working tables (mostly the SELECTS) and do the inserts/updates in my original schema.

Will that decrease the load on the server?

And one off-topic question: Is LA of up to 8.00 healthy for the server ?

Thanks

P.S. If you need any further info, just let me know I'll share it.

Best Answer

PROBLEM #1

You innodb_thread_concurrency is too small. SUGGESTION: It should either be 0 or 64. Please read my post from Aug 14, 2017 (MariaDB 10.1.22 to use more RAM rather than CPU) on why.

PROBLEM #2

I don't see innodb_read_io_threads and innodb_write_io_threads mentioned. By default, both are 4. SUGGESTION: I would double both to 8 (Percona Server's default). Please see my post from Jun 20, 2013(MySQL using too much CPU).

PROBLEM #3

Why is your innodb_log_file_size 30G ? InnoDB Redo Logs are written sequentially. IMHO, writing sequential files on an SSD is not that great (See my post from Feb 06, 2014 : MySQL on SSD - what are the disadvantages? and Sep 25, 2014 : What is the best storing device for DB transaction log?). SUGGESTION #1: You could configure InnoDB to have the Redo Logs on a separate disk. SUGGESTION #2: Please make innodb_log_file_size 1G for now. Then, go figure out the proper size by reading my post from Jan 17, 2013 (MySQL 5.5 - Innodb - innodb_log_file_size higher than 4GB combined?) where I point to two posts from Percona's mysqlperformanceblog:

GIVE IT A TRY !!!

As for the one off-topic: Server with 16GB RAM and LA of 8.00 ??? Of course, not !!! Try these suggestions and see if it really helps.

Related Question