Daily we ship our SQL Server backups across the WAN. We need to minimize the size of these backups so it does not take forever.
We don't mind if our backup process takes a bit longer; as it stands we need to move 30gigs of compressed backup across the WAN that takes over 10 hours.
There are 2 options we have to get smaller daily backups.
- Log shipping, which would mean we would have to restructure DR process.
- Strip information out of the db and rebuild on the other side (drop non clustered indexes, pack clustered indexes at 100% – rebuild on the other side)
Both would involve a fair amount of work from our part. We are using SQL Server 2008 pro, all backups are compressed.
Are there any commercial products that can give us similar backup size to option (2)?
Is there a comprehensive script out there that will allow us to accomplish (2)? (handling indexed views, filtered indexes, foreign keys and so on)
Best Answer
First thought based on comments...
Use differential backups every, say, 6 hours, to reduce the size/time of backup + FTP. Then reduce your full backup + FTP to weekends only. This avoids complexity of log shipping, simple to do, and only adds slight complexity to DR
I feel that differential backups are overlooked... I've suggested using them before:
Using DIFF backups to solve this
Using DIFF backups to speed up a backup/restore server migration
Edit: after jcolebrand's comment I'll attempt to explain more
A differential backup only takes pages that have changed. Outside of any index maintenance (which can affect a lot of the database), only a few % of pages will change during a day. So a differential backup is a lot smaller than a full backup before any compression.
If you have a full backup, say weekly, you can then do daily differentials and ship them off site. A daily full backup with differentials will still require both files off site.
This should solve the problem of getting data from A to B, C and D quickly.
You probably need to restore both the full and latest differential to get the latest data but you can maybe work around this with NORECOVERY and a STANDBY file (I haven't tried it with a diff restore for years since I was last in a pure DBA job).
An added bonus is that diff backups are unrelated to ongoing log backups so you can separate any High Availability/DR requirement from the "get data to the code monkeys" requirement.
I see some issues if you have daily full backups by policy or audit, but the diff restore can be applied before any log restores to shorten recovery time. Unlike backups, diff and log restores do interact.
Hope I've covered most bases...