Mongodb ReplicaSet Changing Config Format

mongodbmongodb-3.4

I have a 3 node mongodb replica set which uses the old syntax (non-Yaml) syntax that I am trying to change to use the new Yaml syntax

old syntax

auth = true
bind_ip = 0.0.0.0
dbpath = /var/lib/mongodb
fork = false
keyFile = /etc/mongod.key
logappend = true
logpath = /var/log/mongodb/mongod.log
nojournal = false
port = 27017
replSet = rs0
rest = false
smallfiles = false

Yaml file

net:
    bindIp: 0.0.0.0
    port: 27017
    http:
        enabled: true
        RESTInterfaceEnabled: true
storage:
    dbPath: /var/lib/mongodb
    mmapv1:
        smallFiles: false
    journal:
        enabled: false
processManagement:
    fork: false
security:
    keyFile: /etc/mongod.key
    authorization: enabled
systemLog:
    logAppend: true
    path: /var/log/mongodb/mongod.log
replication:
    replSetName: "rs0"

I have made the change on the primary but it no longer properly starts up. I get the following error message in the log

SyncSourceFeedback error sending update to :27017:
CallbackCanceled: Reporter no longer valid

I did a search on the error but I have not been able to find a solution.
The replicaset is running mongo v3.4.9

I have also tried the following:

  • Setting up each server with the same configuration
  • Stopping services on all members and starting one at a time.

These attempts did not work

Best Answer

After a few days of looking at this and testing different things, I found that there was a missing configuration value

systemLog.destination: file

I discovered this by running the following command

sudo mongod -f /etc/mongod.conf

instead of doing basic service startup commands

(i.e. sudo service mongod start...)

It showed the following error:

BadValue: Can only use systemLog.path if systemLog.destination is to a file

A couple of other things as you troubleshoot. Make sure you maintain the mongo file and folder permissions. I was deleting log and config files as I was troubleshooting and did not realize it was not allowing the engine to start a few times. I ran the following commands to correct the permissions

sudo chown -R mongodb:mongodb /var/log/mongodb
sudo chown -R mongodb:mongodb /var/lib/mongodb

I also went through the process of removing one of the secondaries from the replicaSet 

rs.remove("<server>:27017")

and 

rs.reconfig()

Then logging directly to the old secondary that was a stand alone made sure I can access the database as a standalone.

I then readded the system during the changes of the config

rs.add({host: "<servername>"})

Here is the final config file:

net:
    bindIp: [0.0.0.0,<host_name>,127.0.0.1]
    port: 27017
    http:
        enabled: true
        RESTInterfaceEnabled: true
storage:
    engine: wiredTiger
    dbPath: "/var/lib/mongodb"
    journal:
        enabled: false
processManagement:
    fork: false
security:
    keyFile: "/etc/mongod.key"
    authorization: "enabled"
systemLog:
    logAppend: true
    destination: "file"
    path: "/var/log/mongodb/mongod.log"

replication:
    replSetName: "rs0"

I hope this helps someone

Related Question