SUGGESTION #1 : Use Distribution Masters
A Distribution Master is a mysql slave with log-bin enabled, log-slave-updates enabled and contains only tables with the BLACKHOLE Storage Engine. You can apply replicate-do-db to the Distribution Master and create binary logs at the Distribution Master that contains only the DB schema(s) you want binlogged. In this way you reduce the size of outgoing binlogs from the Distribution Master.
You can setup a Distribution Master as follows:
- mysqldump your database(s) using --no-data option to generate a schema-only dump.
- Load the schema-only dump to the Distribution Master.
- Convert every table in the Distribution Master to the BLACKHOLE storage engine.
- Setup replication to the Distribution Master from a master with real data.
- Add replicate-do-db option(s) to /etc/my.cnf of the Distribution Master.
For steps 2 and 3 you could also edit the schema-only dump and replace ENGINE=MyISAM and ENGINE=InnoDB with ENGINE=BLACKHOLE and then load that edited schema-only dump into the Distribution Master.
In step 3 only, if you want to script the conversion of all MyISAM and InnoDB tables to BLACKHOLE in the Distribution Master, run the following query and output it to a text file:
mysql -h... -u... -p... -A --skip-column-names -e"SELECT CONCAT('ALTER TABLE ',table_schema,'.',table_name', ENGINE=BLACKHOLE;') BlackholeConversion FROM information_schema.tables WHERE table_schema NOT IN ('information_schema','mysql') AND engine <> 'BLACKHOLE'" > BlackholeMaker.sql
An added bonus to scripting the conversion of table to the BLACKHOLE storage engine is that MEMORY storage engine tables are converted as well. While MEMORY storage engine table do not take up disk space for data storage, it will take up memory. Converting MEMORY tables to BLACKHOLE will keep memory in the Distribution Master uncluttered.
As long as you do not send any DDL into the Distribution Master, you can transmit any DML (INSERT,UPDATE,DELETE) you so desire before letting clients replicate just the DB info they want.
I already wrote a post in another StackExchange site that discusses using a Distribution Master.
SUGGESTION #2 : Use Smaller Binary Logs and Relay Logs
If you set max_binlog_size to something ridiculously small, then binlogs can be collected and shipped out in smaller chunks. There is also a separate option to set the size of relay logs, max_relay_log_size. If max_relay_log_size = 0, it will default to whatever max_binlog_size is set to.
SUGGESTION #3 : Use Semisynchronous Replication (MySQL 5.5 only)
Setup your main database and multiple Distribution Masters as MySQL 5.5. Enable Semisynchronous Replication so that the main database can quickly ship binlogs to the Distribution Master. If ALL your slaves are Distribution Masters, you may not need Semisynchronous Replication or MySQL 5.5. If any of the slaves, other than Distribution Masters, have real data for reporting, high availability, passive standby or backup purposes, then go with MySQL 5.5 in conjunction with Semisynchronous Replication.
SUGGESTION #4 : Use Statement-Based Binary Logging NOT Row-Based
If an SQL statement updates multiple rows in a table, Statement-Based Binary Logging (SBBL) stores only the SQL statement. The same SQL statement using Row-Based Binary Logging (RBBL) will actual record the row change for each row. This makes it obvious that transmitting SQL statements will save space on binary logs doing SBBL over RBBL.
Another problem is using RBBL in conjunction with replicate-do-db where table name has the database prepended. This cannot be good for a slave, especially for a Distribution Master. Therefore, make sure all DML does not have a a database and a period in front of any table names.
The replication setup you have cannot be maintained when upgrding to MySQL 5.5
You will have to go with Master/Slave. Here is the reason:
Someone asked a question about gibberish appearing in a binary log and I answered this question
In my answer, I explained that binary logs start at different positions in different versions of MySQL
Here are those positions:
- 107 for MySQL 5.5
- 106 for MySQL 5.1
- 98 for MySQL 5.0 and back
With replication, these numbers apply to the relay logs as well. The net effect is that a Slave can replicate from an older version of MySQL, but not the other way around.
You are better off doing the following:
- Convert Master/Master to Master/Slave
- Upgrade to MySQL 5.5 on the Slave
- Point all apps at the Slave (now called NewMaster)
- service mysql stop on the Master
- Upgrade to MySQL 5.5 on the Master (now called NewSlave) but do not start mysql
- service mysql stop on NewMaster
- rsync NewMaster's /var/lib/mysql to NewSlaves's /var/lib/mysql
- rm /var/log/mysql/master.info on NewMaster
- rm /var/log/mysql/master.info on NewSlave
- service mysql start --skip-networking on NewMaster
- run RESET MASTER on NewMaster
- service mysql restart on NewMaster
- service mysql start on NewSlave
- run RESET MASTER on NewSlave
- setup Master/Master Replication using position 107 as start master_log_pos
With regard to using yum to upgrade, I run in panic and scream in horror at the thought of it.
Here is my more conservative approach:
- I would mysqldump everything EXCEPT the mysql schema.
Run this to generate SQL for all mysql user grants:
mysql -hhostaddr -umyuserid -pmypassword --skip-column-names -A -e"SELECT CONCAT('SHOW GRANTS FOR ''',user,'''@''',host,''';') FROM mysql.user WHERE user<>''" | mysql -hhostaddr -umyuserid -pmypassword --skip-column-names -A | sed 's/$/;/g' > /root/MySQLGrants.sql
Uninstall MySQL 5.1
- Install MySQL 5.5 from RPMs by hand.
- Load /root/MySQLGrants.sql into MySQL 5.5
- Load the mysqldump'd data back into MySQL 5.5
Best Answer
What could go wrong? If the way MyISAM changed the format of how something is stored in their files, it will break on 5.1. This will render your stats database on the master corrupt and unusable.
Even if the initial load went fine, it doesn't mean something won't come along later with the result listed above...a corrupt database. It's a risk, and if it can be avoided, I personally would do so.
And in your case, it can be avoided by dumping the data using
mysqldump
quickly and then load into the 5.1 instance.