I have a query that takes 15 seconds to get 350 results in a MySQL 5.6 Server and I am unable to diagnose why, I am still very new to database optimizing. U
The EXPLAIN visual does show some non-unique key lookups but each only says one 1 row look up.
The tabular EXPLAIN which I am not able to interpret and I am hoping someone else can here looks like .
I have tried switching the ending LIMIT = 350
to 100, 10 and 1 and the query takes exactly the same amount of time to run, about 15 seconds.
I have tried nixing the views but besides making it hard to recreate this query it did not improve performance.
Perhaps related, in other EXPLAIN statements in our MySQL DB, I've seen a view referenced with Materialized next to it, but that does not appear near next to any of the three views used in this query, in fact I don't even see the views referenced at all instead only the tables they reference. Is that a factor?
My last attempt was replacing the final selected column which is a listlineitems.*
with the specific columns, since I've read that can improve speed and is just better practice, but I get the sense that is not going to dramatically improve this situation.
Here's the query –
SELECT
0 AS 'Check',
DATE_FORMAT(`listlineitems`.`dateEntered`,
'%Y-%m-%d') AS 'Date Entered',
`listlineitems`.`itemId` AS 'parentTableIdx',
`listlineitems`.`parentProjectId` AS 'parentProjectIdx',
`listlineitems`.`idx` AS 'ID',
IF(`listlineitems`.`active` = 1,
'Active',
'Inactive') AS 'Active/Inactive',
CONCAT(`listUsers`.`FirstName`,
' ',
`listUsers`.`LastName`) AS 'Employee',
CASE `listlineitems`.`type`
WHEN 1 THEN 'Time Entry'
WHEN 2 THEN 'Expense Entry'
END AS 'Type',
`listcustomers`.`name` AS 'Customer',
`listlocations`.`name` AS 'Location',
`listareas`.`name` AS 'Area',
`listassets`.`name` AS 'Asset',
`listprojects`.`name` AS 'Project',
`listprojects`.`number` 'Project #',
`listprojects`.`autoassign` 'autoassign',
`listactivities`.`name` AS 'Activity',
(CASE `listlineitems`.`type`
WHEN 1 THEN `listlineitems`.`qty`
WHEN 2 THEN `listlineitems`.`qty`
END) AS 'Quantity',
`listlineitems`.`taxable` AS 'Taxable',
`listlineitems`.`totalAmount` - `listlineitems`.`taxAmount` AS 'Pre-Tax Amount',
`listlineitems`.`taxAmount` AS 'Tax Amount',
`listlineitems`.`totalAmount` AS 'Total Amount',
`listCustomers`.`idx` AS 'parentCustomerIdx',
`listLocations`.`idx` AS 'parentLocationIdx',
`listAreas`.`idx` AS 'parentAreaIdx',
`listAssets`.`idx` AS 'parentAssetIdx',
CONCAT(`listcustomers`.`name`,
'/',
`listlocations`.`name`,
'/',
`listareas`.`name`,
'/',
`listassets`.`name`,
'/',
`listprojects`.`name`) AS 'Path',
IF(`listlineitems`.`customerViewable` = 1,
'Yes',
'No') AS 'Cust. Viewable',
(CASE
WHEN `listlineitems`.`type` = 2 THEN `listexpenseentry`.`TotalCostToPSI` - `listexpenseentry`.`TaxCostToPSI`
ELSE `listlineitems`.`totalAmount` - `listlineitems`.`taxAmount`
END) AS 'preTaxCostPSI',
(CASE
WHEN `listlineitems`.`type` = 2 THEN `listexpenseentry`.`TaxCostToPSI`
ELSE `listlineitems`.`taxAmount`
END) AS 'taxCostPSI',
(CASE
WHEN `listlineitems`.`type` = 2 THEN `listexpenseentry`.`TotalCostToPSI`
ELSE `listlineitems`.`totalAmount`
END) AS 'totalCostPSI',
view_solinx2.lastAltered AS 'lastalteredSO',
view_polinx2.lastAlteredPO AS 'lastalteredPO',
view_invlinx2.lastAlteredInv AS 'lastalteredInv',
view_solinx2.lastAlteredAfterConfirmation AS 'lastAlteredAfterConfirmation',
view_solinx2.roleIdSO AS 'roleIdSO',
view_polinx2.roleIdPO AS 'roleIdPO',
view_polinx2.userIdPO AS 'userIdPO',
view_polinx2.lastAlteredafterConfirmation AS 'lastAlteredAfterConfirmationPO',
view_invlinx2.roleIdInv AS 'roleIdInv',
view_invlinx2.userIdInv AS 'userIdInv',
view_invlinx2.lastAlteredafterConfirmation AS 'lastAlteredAfterConfirmationInv',
view_solinx2.roleId AS 'roleId',
view_solinx2.userId AS 'userId',
view_solinx2.soId AS 'SOId',
view_solinx2.autoassignSO AS 'autoassignSO',
IF(view_solinx2.notNeeded = 1,
'Not Needed',
view_solinx2.number) AS 'SOname',
view_solinx2.dateEntered AS 'SoDate',
view_solinx2.totalSOAmount AS 'SoTotal',
view_invlinx2.invId AS 'InvId',
IF(view_solinx2.notNeeded = 1,
'------',
view_invlinx2.`number`) AS 'InvName',
view_invlinx2.dateEntered AS 'InvDate',
view_invlinx2.amount AS 'InvTotal',
view_polinx2.poId AS 'POId',
IF(view_solinx2.notNeeded = 1,
'------',
view_polinx2.`number`) AS 'POName',
view_polinx2.dateEntered AS 'PODate',
view_polinx2.amount AS 'POTotal',
(SELECT
listsalesorders.number
FROM
listsalesorders
WHERE
listsalesorders.idx = autoassign) AS 'test',
`listlineitems`.*
FROM
`listlineitems`
LEFT JOIN
`listUsers` ON `listlineitems`.`individualId` = `listUsers`.`idx`
LEFT JOIN
`listprojects` ON `listlineitems`.`parentProjectId` = `listprojects`.`idx`
LEFT JOIN
`listassets` ON `listlineitems`.`parentAssetId` = `listassets`.`idx`
LEFT JOIN
`listareas` ON `listlineitems`.`parentAreaId` = `listareas`.`idx`
LEFT JOIN
`listlocations` ON `listlineitems`.`parentLocationId` = `listlocations`.`idx`
LEFT JOIN
`listcustomers` ON `listlineitems`.`parentCustomerId` = `listcustomers`.`idx`
LEFT JOIN
`listactivities` ON `listactivities`.`idx` = `listlineitems`.`activityCode`
LEFT JOIN
`listexpenseentry` ON (`listexpenseentry`.`idx` = `listlineitems`.`itemId`
AND `listlineitems`.`type` = 2)
LEFT JOIN
view_solinx2 ON view_solinx2.idx = listlineitems.idx
LEFT JOIN
view_polinx2 ON view_polinx2.idx = listlineitems.idx
LEFT JOIN
view_invlinx2 ON view_invlinx2.idx = listlineitems.idx
GROUP BY `listlineitems`.`idx`
ORDER BY `listlineitems`.`dateEntered` DESC
LIMIT 10;
Additional Info: There are indexes for customer, location, area, asset, project, dateEntered, itemId, lineitemtype and a few others that I don't think are used (they're columns are not used in the joins, should I IGNORE INDEX for them?).
Also, if it's useful to know, the customer to projects have their own connections. Projects have an asset foreign key, assets have an area foreign key, areas have a location foreign key, and locations have a customer foreign key, all many to one (one customer has many locations, one location has many areas etc).
Any suggestions or a plan of attack/what to look for is greatly appreciated. Any other questions on the structure of the database please just ask. I've been working on this for two days now and have got no where.
Best Answer
Turning comments into an answer…
To start with: Different
LIMIT
s changing hardly anything is due to the fact, all the hard work of collecting and sorting the data has to be done before the limit is applied.In order to find the source of the resource consumption:
ORDER BY
(only) and (then)LEFT
from all theJOIN
s (one at a time).GROUP BY
(and possibly use aWHERE
instead).Please comment, if and as this requires adjustment / further detail.