Refactor the query as follows:
SELECT
readings.*
FROM
(
SELECT boxsn FROM readings
WHERE (time >= 1325404800)
AND (time < 1326317400)
ORDER BY `time` ASC
) readings_keys
LEFT JOIN
(
SELECT id AS boxsn FROM boards WHERE siteId = '1'
) boards
USING (boxsn)
LEFT JOIN readings
USING (boxsn)
;
Make sure you have the following indexes:
ALTER TABLE boards ADD INDEX siteId_id_ndx (siteId,id);
ALTER TABLE readings ADD INDEX time_boxsn_ndx (time,boxsn);
You can drop the other index
ALTER TABLE readings DROP INDEX boxsn_time_ndx;
You should definitely see a dramatic improvement in performance as the tables grow.
In your case,
- The first EXPLAIN plan says you have to perform a lookup of SerialNumber for each row in
readings
against a list of value in memory
- The second EXPLAIN plan says you have to perform a lookup of SerialNumber for each row in
readings
against a table.
UPDATE 2012-01-12 14:03 EDT
I refactored it again to make sure the readings
keys and boards
keys are combined correctly before retrieving the data from the readings
table:
SELECT
readings.*
FROM
(
SELECT A.* FROM
(
SELECT boxsn FROM readings
WHERE (time >= 1325404800)
AND (time < 1326317400)
ORDER BY `time` ASC
) A
LEFT JOIN
(
SELECT id AS boxsn
FROM boards
WHERE siteId = '1'
) B
USING (boxsn)
WHERE B.boxsn IS NOT NULL
) readings_keys
LEFT JOIN readings
USING (boxsn)
;
This is the relational division problem and there is a question about it at SO, with a lot of ways to write this query, plus performance analysis for PostgreSQL: How to filter SQL results in a has-many-through relation
Shamelessly copying code form there and removing/changing code for answers that have features lacking from MySQL, like CTEs, EXCEPT
, INTERSECT
, etc, here are a few ways to do this.
Assumptions:
- the table is called
factors
- there is a
UNIQUE
constraint on (wordid, docid)
- there is a
documents
and a words
table:
Easy to write, medium efficiency:
-- Query 1 -- by Martin
SELECT d.docid, d.docname
FROM document d
JOIN factors f USING (docid)
WHERE f.wordid IN (2, 4, 5)
GROUP BY d.docid
HAVING COUNT(*) = 3 ; -- number of words
Easy to write, medium efficiency:
-- Query 2 -- by Erwin
SELECT d.docid, d.docname
FROM documents d
JOIN (
SELECT docid
FROM factors
WHERE wordid IN (2, 4, 5)
GROUP BY docid
HAVING COUNT(*) = 3
) f USING (docid) ;
More complex to write, very good efficiency in Postgres - probably lousy in MySQL:
-- Query 4 -- by Derek
SELECT d.docid, d.docname
FROM documents d
WHERE d.docid IN (SELECT docid FROM factors WHERE wordid = 2)
AND d.docid IN (SELECT docid FROM factors WHERE wordid = 4);
AND d.docid IN (SELECT docid FROM factors WHERE wordid = 5);
More complex to write, very good efficiency in Postgres - and probably the same in MySQL:
-- Query 5 -- by Erwin
SELECT d.docid, d.docname
FROM documents d
WHERE EXISTS (SELECT * FROM factors
WHERE docid = d.docid AND wordid = 2)
AND EXISTS (SELECT * FROM factors
WHERE docid = d.docid AND wordid = 4)
AND EXISTS (SELECT * FROM factors
WHERE docid = d.docid AND wordid = 5) ;
More complex to write, very good efficiency in Postgres - and probably the same in MySQL:
-- Query 6 -- by Sean
SELECT d.docid, d.docname
FROM documents d
JOIN factors x ON d.docid = x.docid
JOIN factors y ON d.docid = y.docid
JOIN factors z ON d.docid = z.docid
WHERE x.wordid = 2
AND y.wordid = 4
AND z.wordid = 5 ;
Easy to write and extend to an arbitrary set of words
but not as efficient as the JOIN
and EXISTS
solutions:
-- Query 7 -- by ypercube
SELECT d.docid, d.docname
FROM documents d
WHERE NOT EXISTS (
SELECT *
FROM words AS w
WHERE w.wordid IN (2, 4, 5)
AND NOT EXISTS (
SELECT *
FROM factors AS f
WHERE f.docid = d.docid
AND f.wordid = w.wordid
)
);
Easy to write, not good efficiency:
-- Query 8 -- by ypercube
SELECT d.docid, d.docname
FROM documents d
WHERE NOT EXISTS (
SELECT *
FROM (
SELECT 2 AS wordid UNION ALL
SELECT 4 UNION ALL
SELECT 5
) AS w
WHERE NOT EXISTS (
SELECT *
FROM factors AS f
WHERE f.docid = d.docid
AND f.wordid = w.wordid
)
);
Enjoy testing them :)
Best Answer
Normally I would recommend against using a function in a search (UPPER()), but since the wildcard is there, the expression is non-sargable in any case!
The db-fiddle is here!