MongoDB – Does local.oplog.rs Hold All Data in MongoDB Replication?

backupmongodb

I'm trying to do backup full and incremental backup, I wonder does local.oplog.rs collecitons contain all info that I need to do backup and incremental backup?

Best Answer

The MongoDB oplog is a special capped collection including all data used for replication. The oplog alone only includes an idempotent log of data changes. You can use the oplog for incremental backup and point-in-time restoration for a replica set, but note that an oplog backup is only valid to apply against an existing full data backup of the same replica set with a common oplog entry.

If you want to backup the oplog.rs collection independent of the rest of the data files you will have to use mongodump, which can have a significant effect on a production deployment as all of the oplog data needs to be read into memory by mongodump. The oplog.rs collection also has a special caveat on indexing: it is designed to be scanned in $natural order and does not support adding any indexes.

A typical incremental backup approach for a production system is using filesystem snapshots assuming your deployment (and filesystem) meet the requirements for a consistent backup. Filesystem snapshots capture the changes since the last snapshot, so are incremental after the first snapshot. Ongoing snapshots are less impactful for an active deployment than mongodump.

Another common approach is using agent-based backup (for example, MongoDB Cloud Manager or MongoDB Ops Manager). Agent-based approaches generally also use the oplog for real-time backup. See Cloud Manager Backup Preparations for some background.

The MongoDB manual is the best reference for supported backup methods for your version of MongoDB server. Backing up using mongodump for the oplog alone is not a recommended (or documented) backup method.