I have a large daily data feed which can contain up to around 5,000 ID's. I need to delete all pre-existing records in the DB that are not referenced in the new feed. Should I create a temporary table for these 5,000 ID's, and run a LEFT JOIN IF NULL DELETE statement? Or, is it all the same if I just pass the ID's over and run a DELETE IF NOT IN statement. My assumption is that MySQL would do the work of creating the temp table for me when using IN statements.
Mysql – Large NOT IN Statement vs Creating/Populating Temp Table
MySQLtemporary-tables
Related Question
- Sql-server – Temp Table Clustered Key Not Being Honored: Bug or Expected Functionality
- Mysql – DELETE … IN hangs after deleting half of a large table (even for deleting 10 rows)
- Mysql – thesql memory temp table creation SLOW
- Mysql – Create Range Partition on existing large MySQL table
- Sql-server – Should I use a temp table or join
Best Answer
With a large set you may want to use an
NOT EXISTS
clause rather than aNOT IN
clause. It will help if you index the id on your feed table. Using an index will allow the optimizer to use the index rather than reading the table. Try a query like: