Does anyone know of any existing tools/products that accomplish what I'm trying to do?
After searching around for a bit the only thing I could find surrounding what I'm trying to accomplish is someone else looking for the same thing http://forums.mysql.com/read.php?24,192250,192250
The idea is I'd like to capture all the traffic to my master to save for a replay log against a snapshot of the entire database taken when the monitoring started. Bin logs won't serve what I want since they only include writes. I want read activity to realistically view the effects of proposed changes with "real" production traffic.
Real production traffic meaning everything from all applications that are hitting the database to be modified. If there's some application level change, tests running just that app don't account for other activity going on in the system at the time. Running all applications in a test environment aren't guaranteed to have the same state of the database. I could take a test snapshot as a starting point as I fire them up but the applications don't have the ability to do an exact replay of their own activity.
I've use the tcpdump script from http://www.mysqlperformanceblog.com/2008/11/07/poor-mans-query-logging/ to monitor activity but this doesn't tell me which queries are coming from which connections. Part of the playback I'm wanting is a multi threaded approach that replays the activity from the same number of threads that were actually in use.
I can't afford to turn on general query logging b/c my production master wouldn't be able to handle the performance hit for that.
The whole snapshot part of the process is to have a golden start point database to ensure everything is the same during the start of each test run.
Best Answer
I believe that pt-query-digest from the Percona Toolkit (http://www.percona.com/doc/percona-toolkit/2.0/pt-query-digest.html) is what you are looking for.