Mysql – Creating Archiving database for old data in MySQL

If you are sure the Slave is sync'd with the Master, you could do the following on the Slave:

• Activate Binary Logging
• Perform a full mysqldump every three months
• Run FLUSH LOGS; every midnight
• After the FLUSH LOGS;, make a physical copy of new binary logs, except the last one
• OPTIONAL Set expire_logs_days=180 in my.cnf and run SET GLOBAL expire_logs_days=180; (I used 180 because this give you 90 days to have access to changes from the previous two quarters. If you produce more than one binary log per day, make sure you have enough diskspace to hold 180G of binary logs or just reduce the number from 180. The idea is to have the changes to the Master. They are stored already in the Master's Binary Logs. Those same changes can be on the Slave as well if you activate Binary Logging on the Slave. The Master never needs to feel anything throughout this process.

You could use pt-table-checksum and pt-table-sync to sync up the Slave from the Master. (I wrote a shell script to automate them using mk-table-checksum and mk-table-sync)

Doing these things will allow you to reload a snapshot of the beginning of a Quarter onto another server. Replay all logs on the Slave (using mysqlbinlog) to any datetime from those logs to view different days within the next Quarter.

If all your data is InnoDB, here is an Alternative from the Master

• Binary Logging on the Master should already be enabled due to Replication
• Perform a full mysqldump (using --single-transaction) every three months
• Run FLUSH LOGS; every midnight
• After the FLUSH LOGS;, make a physical copy of new binary logs, except the last one
• OPTIONAL Set expire_logs_days=180 in my.cnf and run SET GLOBAL expire_logs_days=180; (I used 180 because this give you 90 days to have access to changes from the previous two quarters. If you produce more than one binary log per day, make sure you have enough diskspace to hold 180G of binary logs or just reduce the number from 180.

These are just some ideas you can try to organize using the Slave or the Master.

When we did something similar recently ~250GB, we first used an EC2 slave from production database as a staging slave with SSL and binary logging enabled -- to make table dumps, etc. Converting any tables to InnoDB if you need here. Load data dumps from EC2 to RDS. Then used a binary log feeding strategy to load data and keep it current. At cutover time, flush the last binary log-- and load it into RDS and you're done. I prepped the logs first (removing any set cmds that require super) and just fed them to the mysql client on rds. Next time I want to try Tungsten replicator.