Mysql – Deleting old rows when a new one is added and there are more than 100 rows

MySQLtrigger

For a school assignment we need to keep the number of rows in a mysqldb under 100. The newest rows must stay. I've tried using a trigger but with no success.

CREATE TRIGGER delete_rows 
AFTER INSERT ON `check`.log 
FOR EACH ROW 
BEGIN 
   IF(SELECT COUNT(*) FROM `check`.log)>100 
   THEN 
      DELETE FROM `check`.log LIMIT 1; 
   END IF; 
END

This resulted in the following error:

ERROR 1442 (HY000): Can't update table 'log' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.

Then I tried monitoring /var/lib/mysql/check/log.ibd with incron and let it execute a script. when opening log.ibd with nano, incron executes my script but when I use mysql to insert something into it, nothing happens.

The incron instruction:

/var/lib/mysql/check/log.ibd IN_ALL_EVENTS /etc/scripts/test

can somebody please help me.

Best Answer

Your trigger is not specifying which row to delete and is only limiting it to deleting a single record. You are going to want to update the trigger to specify which record to delete and I would guess that you want to delete the oldest one.

https://dev.mysql.com/doc/refman/5.6/en/delete.html

The LIMIT clause places a limit on the number of rows that can be deleted

SUGGESTION #1 : Use Stored Procedure for Specialized DML

Having to micromanage the number of DELETEs per user to mitigate DML should not become a server-centric function for DELETEs against a specific table. That should remain an application-centric function. Therefore, create a Stored Procedure that can manage the number of DELETEs.

SUGGESTION #2 : Use the BLACKHOLE storage engine

Any audit trail of DELETEs using a real table can create a localized disk bottleneck that would be a little clumsy to throttle. If the audit trail was recorded in a BLACKHOLE table, that would significantly reduce any disk I/O issues that would otherwise rear its ugly. If you write to a BLACKHOLE table, where would the audit trail data be written? That's leads up to...

SUGGESTION #3 : Use MySQL Replication

How would MySQL Replication help? Suppose the table is called mydb.audittrail. If the mydb.audittrail uses BLACKHOLE on the Prod Server, it could replicate to a Slave that that has mydb.audittrail as MyISAM. That would be the location of the actual audit data. The only drawback would be added disk I/O for writing audit trail info to the binary logs. You can further reduce that by having the binary logs stored on SSD or a RAM disk.

Please see my answer to Is it normal to use many Triggers? for further info on this concept of audit trail recording with the BLACKHOLE Storage Engine and MySQL Replication.

Mysql – How does thesqldump take place when there are more than one database

Let me address each of your three Questions

QUESTION #1

Are the tables of all databases in the server locked during the mysqldump?

ANSWER TO QUESTION #1

Depends on the default settings you allow and what setting you use to override. The --opt parameter is enabled by default. That enables --add-drop-table, --add-locks, --create-options, --quick, --extended-insert, --lock-tables, --set-charset, and --disable-keys.

If you have a database with all InnoDB tables or a mix of InnoDB tables and read-only MyISAM tables, you can just use --single-transaction. This will create a clean point-in-time snapshot of all tables involved in the mysqldump. Any INSERT/UPDATE/DELETE queries against an InnoDB table will not interfere with the point-in-time consistency of the mysqldump.

By the way, --single-transaction is mutually exclusive to --lock-all-tables. Thus, using --single-transaction will disable --lock-all-tables.

QUESTION #2

If I have to do an incremental backup using mysqlbinlog from the last full backup, should I take the time when mysqldump started or when it ended?

ANSWER TO QUESTION #2

You could take the time if you do mysqlbinlog at different times of the day. You could save yourself some off that housekeeping information by simply running FLUSH LOGS; or FLUSH BINARY LOGS; the same time every night. For example, you could script a cronjob to run FLUSH LOGS;at 11:59 PM like this

#!/bin/sh
sleep 55
mysql -uroot -p... -e"FLUSH LOGS;"

That way you can logically group together all binary logs that have the same date when formulating any needed incremental backup.

You do not have record times or positions in this respect: If you include the option --master-data=2 in the mysqldump, the log file and position at recorded as a comment on line 22 of the mysqldump output.

QUESTION #3

What happens when one is using the database when mysqldump is running? How can I take the start time for mysqlbinlog? What happens when mysqldump is executed?

ANSWER TO QUESTION #3

Here is where you have to be very careful. As I mentioned earlier, if you have a database with all InnoDB tables or a mix of InnoDB tables and read-only MyISAM tables, you can use --single-transaction and every table involved in the mysqldump will exist in the same point-in-time.

What Can Go Wrong #1

If you are performing INSERT/UPDATE/DELETE queries against a MyISAM table during the mysqldump, then the MyISAM tables will not be point-in-time consistent.

What Can Go Wrong #2

If you are performing ALTER TABLE, DROP TABLE, RENAME TABLE, or TRUNCATE TABLE against any table (MyISAM or InnoDB), consistent mysqldumps are no longer isolated. The transaction you issued with --single-transaction is released (made null and void) and all tables afterward can no longer the point-in-time consistent.

If one of these occur, recording start times or position is rendered useless.