MySQL – Optimized Query to Fetch a Record from 10 Million Entries

MySQLoptimizationperformancequery-performanceselectunique-constraint

A table has 2 fields

| UniqueKey |  TimeStamp |
-------------------------
| xfsddddq  | 1024125412 |
| xfstttdx  | 1024125413 |<
| xfsdxxau  | 1024125415 |

Estimated, there are 10 million records. Need to verify if UniqueKey is present from within (CURRENT_TIME - 5MINS).


Currently

  • Flushing my table every hour (cannot be decreased further).
  • Inserting more than 10 million records in an hour, as time increases the number of rows executed to fetch one record is also increased further leading to increase in execution time.
  • How to limit query to check one record from (CURRENT_TIME – 5MINS) or effectively fetch the result so that, the time of execution is same at 5th minute and 59th minute.

Best Answer

For the table mydb.mytable with UniqueKey and timeStamp, to see if the UniqueKey exists within the last 5 minutes, simply run this

SELECT COUNT(1) FROM mydb.mytable
WHERE UniqueKey = ????
AND timeStamp >= ( NOW() - INTERVAL 5 MINUTE );

or

SELECT COUNT(1) FROM mydb.mytable
WHERE UniqueKey = ????
AND timeStamp >= ( NOW() - INTERVAL 300 SECOND );

What does the value indicate ???

  • If you get 0, the UniqueKey is older that 5 minutes
  • If you get 1, the UniqueKey is within the 5 minutes

Make sure the UniqueKey has a unique index.

Give it a Try !!!

Since your timestamp is a UNIX timestamp, I'll adjust the code using UNIX_TIMESTAMP() function

SELECT COUNT(1) FROM mydb.mytable
WHERE UniqueKey = ????
AND timeStamp >= UNIX_TIMESTAMP(NOW() - INTERVAL 5 MINUTE);