Mongodb – Incremental backup MongoDB from oplog

backupmongodbmongodb-3.2python

I was trying to take backup using oplog based on ts using below code 

 import pymongo
    import traceback
    from pymongo.cursor import CursorType
    c = pymongo.MongoClient('mongodb://localhost:28001')
    oplog = c.local.oplog.rs
first = next(oplog.find().sort('$natural', pymongo.DESCENDING).limit(-1))
ts = first['ts']
while True:
    cursor = oplog.find({'ts': {'$gt': ts}}, cursor_type=CursorType.TAILABLE_AWAIT, oplog_replay=True)
    while cursor.alive:
        for doc in cursor:
            ts = doc['ts']
            print doc['ns']
            db,collection=str.split(str(doc['ns']),'.',1)

It is working well when we have less opLog size but as i am using it on production where oplog size is 48 GB around it is not working.

I cannot alter oplog size.Please suggest some way to take incremental backup.

Best Answer

Incremental backup procedure

  • lock writes on a secondary member

  • Dump oplog from the recorded oplog position on full (or latest incremental ) backup:

    mongodump --host <secondary> -d local -c oplog.rs -o /mnt/mongo-test_backup/1 
--query '{ "ts" : { $gt :  Timestamp(1437725201, 50) } }'

  • Record latest oplog position (same way as for full backups)

  • Unlock writes

Click Here for detailed information

Related Question