Mongodb – mongo db-Either all host names in a replica set configuration must be localhost references, or none must be; found 1 out of 2

This will be hard to diagnose without the full logs, but the easiest way to get that secondary back to normal is to resync it from scratch from the primary. Basically stop that mongod, delete all data in the dbpath (including subdirectories), then restart. It will kick off the sync process as if it were a new member.

For what it's worth, running two full instances (primary/secondary) on what sound like m1.large instances is probably going to see a lot of resource contention, especially on the disk side of things.

In terms of the crash, based on the limited evidence, it could be the OOM Killer at work:

http://www.mongodb.org/display/DOCS/The+Linux+Out+of+Memory+OOM+Killer

To avoid it, make sure you have some swap space configured.

You need to basically reset the config, which is stored in the local database, and then start from scratch. Assuming you still just have the one server configured as above, restart the instance without the --replSet option, then drop the local.system.replset collection. Once that is done you may restart again with --replSet and you can re-run rs.initiate() but this time pass in the configuration document instead of letting it auto-discover, something like this:

rs.initiate({"_id" : "my-rs0", "version" : 1, "members" : [{"_id" : 0, "host" : "my-server04:27017"}]})

You can also solve this by making sure that the name resolves correctly and so is automatically populated by the default rs.initiate() command, but the above removes any doubts.