MySQL Temporary Tables

MySQLstored-procedurestemporary-tables

I am facing Memory issue in our server. While debugging I came to know that, many number (millions) of temporary tables are created. Is there any way to identify which databases are creating this temporary tables? We have around 50 databases and many of the databases using create temporary table statement in its procedures. In /tmp directory I could see the .ibd, .frm and few .MYI & MYD files. Is it possible to find which database contributing more to creation of temporary tables? Please let me know, if you need any other information.

Thanks in advance.

Best Answer

Not quite what you asked for, but this query may help:

mysql> SELECT db, name FROM mysql.proc WHERE body LIKE '%temporary%';
+---------------+-----------------------------------+
| db            | name                              |
+---------------+-----------------------------------+
| common_schema | rdebug_compile_routine            |
| common_schema | _get_sql_dependencies_internal    |
| common_schema | query_checksum                    |
| common_schema | _split_cleanup_dependency_tables  |
| common_schema | _split_generate_dependency_tables |
+---------------+-----------------------------------+

If /tmp is in a separate filesystem, you are cruisin' for a bruisin'. Change tmpdir to point to some directory in the main FS.

YES.

You can't get a consistent backup even for a single MyISAM table unless you do some type of locking to prevent writes. Rolando gave a pretty thorough answer with options available to you for locking.

One other option that people use for backing up MyISAM data: LVM snapshots. See http://www.lenzg.net/mylvmbackup/ for a great tool to assist with this.

The final recommendation is to stop using MyISAM, and use InnoDB instead. Then you can do fast, non-locking physical backups with Percona XtraBackup.

Re your comment:

Because reading through a large file isn't instantaneous or atomic. While your backup is progressing through the table, other concurrent updates could change both rows that your backup has already read, and rows that your backup hasn't reached yet.

Take a textbook example for transaction behavior: I do a bank transfer by debiting my bank account and crediting your bank account. My bank account is stored on a row that is physically early in the file, and your bank account is stored on a row later in the file.

While this is going on, the backup is reading through the file, and it has read up to a mid-point at the time our transaction happens. When we restore, we get my original account balance, without the debit applied, because the backup had already passed that point when the debit occurred (and it isn't able to go backwards). But the restore includes your updated account balance, because the backup got to that point in the file after we increased your balance.

Ergo, free money! ;-)

MyISAM solves this by requiring the table to be locked against updates while it's doing a read.

InnoDB solves this by keeping multiple versions of rows, for as long as a reading transaction needs to see them for the sake of a consistent view of the database. So anyone can update the data without waiting, even though the backup is in progress.

Percona XtraBackup solves this in a slightly different way: it can go backwards, in a way. While it's reading the data file, it keeps checking the transaction log continually, to see if there are any late changes it needs to include. These changes may apply to parts of the datafile that Percona XtraBackup has already read. But as long as it gets the data file plus any changes that were logged since the backup started, it can reconstruct the full database.

But that only works for storage engines like InnoDB, that create a reliable transaction log. Percona XtraBackup can also back up MyISAM, but only by using locking, like any other backup tool.

Best Answer

Related Solutions

MySQL Backup Breaks a Table Every Time – Solutions

MySQL – Does MyISAM Table Lock During Copy While Running?

YES.

Related Question