Mongodb – Getting empty folder when using mongodump to back up the MongoDB

Some thoughts....

Typically one does not want to store pieces of tightly interrelated information in different systems. The chances of things getting out of sync is significant and now instead of one problem on your hands you have two. One thing you can do with Mongo though is use it to pipeline your data in or data out. My preference is to keep everything in PostgreSQL to the extent this is possible. However, I would note that doing so really requires expert knowledge of PostgreSQL programming and is not for shops unwilling to dedicate to using advanced features. I see a somewhat different set of options than you do. Since my preference is not something I see listed I will give it to you.

You can probably separate your metadata into common data, data required for classes, and document data. In this regard you would have a general catalog table with the basic common information plus one table per class. In this table you would have an hstore, json, or xml field which would store the rest of the data along with columns where you are storing data that must be constrained significantly. This would reduce what you need to put in these tables per class, but would allow you to leverage constraints however you like. The three options have different issues and are worth considering separately:

hstore is relatively limited but also used by a lot of people. It isn't extremely new but it only is a key/value store, and is incapable of nested data structures, unlike json and xml.

json is quite new and doesn't really do a lot right now. This doesn't mean you can't do a lot with it, but you aren't going to do a lot out of the box. If you do you can expect to do a significant amount of programming, probably in plv8js or, if you want to stick with older environments, plperlu or plpython. json is better supported in 9.3 though at least in current development snapshots, so when that version is released things will get better.

xml is the best supported of the three, with the most features, and the longest support history. Then again, it is XML.....

However if you do decide to go with Mongo and PostgreSQL together, note that PostgreSQL supports 2 phase commit meaning you can run the write operations, then issue PREPARE TRANSACTION and if this succeeds do your atomic writes in Mongo. If that succeeds you can then COMMIT in PostgreSQL.

As the page you linked implies, any point in time snapshot technique that includes the data files and the journal will suffice, LVM is just one option. EBS snapshots in Amazon EC2 will also work, as will similar snapshot solutions on SAN, NAS etc. You are not limited to LVM, but that is generally a solution people can implement themselves.

In terms of whether you can copy files to perform a backup, the answer is yes, but only if you stop all writes to the node you are backing up (thereby guaranteeing no changes to the files during the copy). You can do this in a couple of ways:

The most straight forward way is to just shut the node (this should be a secondary) down, copy the files, then start the node back up and let it catch up to the primary (check optime using rs.status()). Rinse and repeat (if you wish) to cycle through all nodes in the set, though the nodes are all identical, so one copy should generally be enough.

The second way (mentioned by sysadmin1138) is to fsync (flush data to disk) and lock (prevent writes) the node but leave it running using the fsyncLock command (again, this should be a secondary). Once you have completed the copy, you unlock the database using the fsyncUnlock command. There are dangers inherent in this technique - for example (and particularly if you are using authentication), you should always lock and unlock on the same connection, otherwise you risk locking yourself out of the database and having to kill the process to recover.

As for other risks, it is common to use a hidden node for backups in each case, which prevents accidentally attempting reads from the node while it is behind, and/or while it is locked (depending on your methods).

Finally, there is one further (paid) option - MMS Backup. This service will essentially do all this for you (for a fee) and give you extras like point-in-time recovery and more - note: I work for MongoDB so I won't give you the hard sell here, but feel free to evaluate it yourself.