MySQL – Timeout on INSERT Lasts Almost 60 Seconds

deadlockMySQL

I have this table:

CREATE TABLE `ClusterDescription` (
  `ClusterDescriptionId` int(11) NOT NULL AUTO_INCREMENT,
  `ClusterId` int(11) NOT NULL,
  `RightBarDateTime` datetime NOT NULL,
  `InstrumentId` int(11) NOT NULL,
  `RowCreated` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`ClusterDescriptionId`),
  UNIQUE KEY `ClusterInstrumentId` (`ClusterId`,`InstrumentId`),
  KEY `InstrumentId` (`InstrumentId`) USING BTREE,
  KEY `OrderDesc` (`RightBarDateTime` DESC,`ClusterId` DESC)
) ENGINE=InnoDB AUTO_INCREMENT=3055989 DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci 

The table has more than 3m rows.

I receive regularly this error:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when
trying to get lock; try restarting transaction

for this command:

INSERT INTO ClusterDescription (ClusterId, RightBarDateTime, InstrumentId)
 SELECT IFNULL(MAX(ClusterId) + 1, 1), :rightBarDateTime, :instrumentId
 FROM ClusterDescription WHERE InstrumentId = :instrumentId
  • There can be up to 50 above INSERT at almost the same time.
  • :instrumentId is never the same in any of the parallel commands.
  • Checking the process list I see lot of these commands waiting to be executed with a Time value > 30 seconds.
  • I set innodb_lock_wait_timeout to 360 seconds which improved the situation. Before I received lot of the above mentioned error and this error:

SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try
restarting transaction

My questions:

  • What is wrong with this table definition?

  • Why do need the parallel INSERT commands more than 50 seconds to be executed?

  • Why do I receive a dead lock error when only 1 table is affected?

Running on MySQL 8.0.13

I need the InnoDB engine because of the descending index.

Best Answer

Both of these improve the efficiency of the SELECT MAX... in the INSERT..SELECT:

Plan A

UNIQUE KEY `ClusterInstrumentId` (`ClusterId`,`InstrumentId`),
KEY `InstrumentId` (`InstrumentId`, ClusterId) USING BTREE,

This assumes you need to look up things by ClusterId.

Plan B

UNIQUE (`InstrumentId`, ClusterId)

This covers the Uniqueness constraint