For many people, the MySQL Achilles' heel is implicit commit.
According to Page 418 Paragraph 3 of the Book
the following commands can and will break a transaction
ALTER TABLE
BEGIN
CREATE INDEX
DROP DATABASE
DROP INDEX
DROP TABLE
RENAME TABLE
TRUNCATE TABLE
LOCK TABLES
UNLOCK TABLES
SET AUTOCOMMIT = 1
START TRANSACTION
SUGGESTION
When it comes to MySQL, any ContinuousIntegration (CI)/SelfService jobs you construct should always make Transactional jobs and DDL scripts mutually exclusive.
This gives you the opportunity to create paradigms that would
- support transactions that are properly isolated with
START TRANSACTION/COMMIT
blocks
- control of DDL by scripting the DDL yourself, running such DDL as either constructor or destructor
- Constructor : DDL to make Tables with a New Design
- Destructor : DDL to make Tables Revert Back to Previous Design
- never combine these operations under one job
WARNING : If you are using MyISAM for any this, you can (un)kindly add MyISAM to the list of things that can break a transaction, maybe not in terms of implicit commit, but definitely in terms of data consistency should a rollback ever be needed.
WHY NOT LVM ?
LVM snapshots are great and restoring whole instances of databases without having to perform heavy SQL processing is ideal. However, when it comes to MySQL, you have to account for two storage engines: InnoDB and MyISAM.
All-InnoDB Database
Look at InnoDB's Architecture (Picture courtesy of Percona CTO Vadim Tkachenko)
InnoDB has many moving parts
- System Tablespace
- Data Dictionary
- Double Write Buffer (support data consistency; used for Crash Recovery)
- Insert Buffer (Buffers Changes to Secondary Non-Unique Indexes)
- Rollback Segments
- Undo Space (where the most uncontrolled growth can happen)
- InnoDB Buffer Pool
- Dirty Data Pages
- Dirty Index Pages
- Changes to NonUnique Indexes
- Other Important Memory Caches
Taking an LVM snapshot of an all-InnoDB database with uncommitted changes floating in the Buffer Pool and Memory caches would yield a dataset that would require InnoDB crash recovery once the LUN is restored and mysqld started.
SUGGESTION FOR ALL-InnoDB
If You Can Shutdown MySQL Before Taking Snapshot
- Run
SET GLOBAL innodb_fast_shutdown = 0;
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 0;
- Run
SHOW GLOBAL STATUS LIKE 'Innodb_buffer_pool_pages_dirty';
- Repeat Step 3 until Innodb_buffer_pool_pages_dirty is 0 or as close to 0 as possible
service mysql stop
- Take LVM snapshot
service mysql stop
If You Cannot Shutdown But Take Snapshot With MySQL Live
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 0;
- Run
SHOW GLOBAL STATUS LIKE 'Innodb_buffer_pool_pages_dirty';
- Repeat Step 2 until Innodb_buffer_pool_pages_dirty is 0 or as close to 0 as possible
- Take LVM snapshot
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 75;
All-MyISAM Database or InnoDB/MyISAM Mix
MyISAM, when accessed, maintains a count of open file handles against it. If MySQL crashes, any MyISAM table with an open file handle count > 0 will be marked as crash and in need of repair (even if nothing is wrong with the data).
Taking an LVM snapshot of a database that has MyISAM tables in use will have one or more MyISAM tables in need of repair when the snapshot is restored and mysqld started.
SUGGESTION FOR All-MyISAM or InnoDB/MyISAM Mix
If You Can Shutdown MySQL Before Taking Snapshot
- Run
SET GLOBAL innodb_fast_shutdown = 0;
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 0;
- Run
SHOW GLOBAL STATUS LIKE 'Innodb_buffer_pool_pages_dirty';
- Repeat Step 3 until Innodb_buffer_pool_pages_dirty is 0 or as close to 0 as possible
service mysql stop
- Take LVM snapshot
service mysql stop
If You Cannot Shutdown But Take Snapshot With MySQL Live
You could enforce a flushing of certain InnoDB tables
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 0;
- Run
SHOW GLOBAL STATUS LIKE 'Innodb_buffer_pool_pages_dirty';
- Repeat Step 2 until Innodb_buffer_pool_pages_dirty is 0 or as close to 0 as possible
- Run
FLUSH TABLES innodb_tbl1,... FOR EXPORT;
on critical InnoDB tables
- Run
FLUSH TABLES WITH READ LOCK;
- Take LVM snapshot
- Run
UNLOCK TABLES;
- Run
SET GLOBAL innodb_max_dirty_pages_pct = 75;
Could MySQL Replication Help ?
While you could restore one LVM snapshot to two servers and setup MySQL Master/Slave Replication, that becomes an extra source of housecleaning when restoring snapshots.
If you run CI jobs on a Master and those jobs are small, replication could be a timesaver under certain circumstances. You could just run STOP SLAVE;
on the Slave, kick off the CI jobs on the Master, and run START SLAVE;
on the Slave when the Master's data is certified.
If the CI jobs alert too much data, you could restore LVM snapshot and setup replication from scratch. If you find yourself doing this often, you could probably do with setting up MySQL Replication.
FINAL THOUGHTS
- It is best to use multiple DB Servers (3 or more) to perform restores and regression tests.
- Convert remaining MyISAM tables to InnoDB if those tables do not need to remain MyISAM.
- If your data content is sensitive, you should execute a CI job to scrub the data after restoring a snapshot before initiating any tests. As an alternative, you may want to take snapshots of MySQL with the data already scrubbed.
Vertica (actually, I think most if not all databases) behaves in this manner because the part that processes the BEGIN TRANSACTION
SQL statement, namely the query parsing and execution engine, is not aware of the client AUTOCOMMIT setting. On the other hand, the client is not aware of the meaning of the string 'BEGIN TRANSACTION'.
The flow of control looks something like this, assuming the command line client vsql
is used to run the example:
- User starts
vsql
and connects to the database.
- User enters
\set AUTOCOMMIT on
. After that moment the client (vsql
) knows to issue an implicit COMMIT
statement after each successful statement.
- User enters
BEGIN TRANSACTION
. vsql
does not recognize that string as one of its internal commands and sends the string to the server for processing.
- Server recognizes the string as a valid SQL statement, compiles it, starts a transaction, and returns a successful result code to the client. Since autocommit is a client-side setting, the server has no idea about it.
- Client, upon receiving the successful result code, issues
COMMIT
to the server.
- Server complies and commits the transaction it has just started.
etc.
Best Answer
No not as such, unless someone is doing testing I have hardly seen people manually doing a rollback. The rollback is specified in stored procedures or transactions in which you either want complete change to be entered or nothing at all.
Depending on how transaction is initiated, if it is under explicit begin transaction and commit yes it will rollback completely to its initial state after the query fails, while if the transaction is not under begin transaction and commit it will store the changes done till the query failed and from after the query failed the it would be stopped from making further changes.
A classic case where system initiates a rollback is when a deadlock happens and SQL Server by internal mechanism chooses deadlock victim, kills the transaction and it may or may not rollback depending on whether it was explicit or implicit transaction. Like when normal select statement is killed their is no rollback as such.