datafilemongodbnosqlperformance
Are there any benefits in disabling mongodb journaling besides performance gain?
I know journaling provides write durability and crash resiliency. But I can tune syncdelay parameter to make data files flush more frequently, but mongodb docs recommends against this. What effect would it have on my mongodb setup and why should I not set syncdelay on production setups?
Thanks,
Abhi
MongoDB's dynamic schema supports different fields for documents within the same collection, so there is no strict requirement for all documents to have an active field. The main consideration here will be how that affects your application logic; it may be simpler to have an explicit field present and check the value.
If you have an index on a field, documents that do not have that field present will have a null value included in the index by default. This can potentially be useful if you want to save some bytes of storage by assuming that missing values are false.
If it only makes sense to store a true value for your use case (and you do not want to search on false values), you can:
$set the active field to indicate true values$unset the active field to indicate false valuesIf you need to query on both true and false values of active, you would be better using a normal (non-sparse) index.
Notes:
You can use the $exists operator to match documents where a field does (or does not) exist.
It doesn't matter if it's a wiredTiger or MMAPv1, they are just storage engines. Whether you lose updates or not depends on write concerns.
Your app communicates with MongoDB server that writes mostly to memory that has a cache of pages which is periodically written and read from the persistent disk. There is also a journal that logs everything.
When your write to the database it writes both to pages and journal simultaneously. Pages will write to the disk depending on memory pressure. By default you don't wait for acknowledgement from the journal because the journal may not write to the disk for a while, which is represented by j=false. And by default w=1 which means it only acknowledges write to the cache pages.
Default: w = 1, j = false - this is fast, but if something happens before the journal has a chance to write to the disk then the data is lost. What's worse is that it will show as if the data has been written/persisted if you rely on w = 1.
w = 1, j = true - in this case you wait for the acknowledgement from the journal that it has written to the disk. This is a lot slower, but ensures the data is persisted.
Best Answer
The only other marginal benefit is if you are using a 32-bit build (which is also not recommended for production use). Since 32-bit builds are limited to ~2Gb of addressable data for memory mapped files, the journal is off by default to allow for more data (with journal enabled, the 32-bit data limit is halved to ~1Gb). In all other cases, the journal is enabled by default. For 64-bit systems, the addressable virtual memory is not a limiting factor for data size.
Any performance gain from disabling the journal is a tradeoff for data & operational risk. I would strongly advise against disabling the journal unless your data is ephemeral (i.e. you don't care about losing it or can easily recreate it from another source of truth).
If you are concerned about possible contention or overhead from disk I/O of journaling, you could always move the journal to a separate volume. NOTE: moving the journal to a separate volume may also change your backup strategy, particularly if you were relying on filesystem snapshots to take a comparatively quick backup without having to
fSyncLock()and quiesce your database.Writes to the journal are fast append-only operations. As you've noted, these provide durability and crash resiliency. If MongoDB stops unexpectedly, it can recover from the journal and data will be in a consistent state.
Without the journal, your data is in an unknown state after an unexpected shutdown and you need to validate it by running repair or (in the case of a replica set node) resync from a known-good copy of the data.
The
syncdelaysetting controls the frequency where the in-memory view is synced with the on-disk view of data (aka the "background flush interval").By default, the
journalCommitIntervalis 100ms and thesyncdelay(background flush) happens every 60 seconds.If you adjust the
syncdelaytoo low, you can end up creating a lot of additional I/O (and reduced performance) because writes to dirty pages can no longer be effectively batched together and the same page will be re-written multiple times. If you adjust thesyncdelayhigher, you can create I/O spikes (and reduced performance) with a large volume of changes being committed in a single batch.The
syncdelayandjournalCommitIntervalparameters are available as tuneable settings because in some cases it may make sense to adjust these (for example, to try to temporarily help an under-provisioned I/O system by reducing write spikes). In a healthy production system, it would be best to leave these settings as-is.For more information, How MongoDB’s Journaling Works is a helpful read.