mongodbwiredtiger
We have a cluster of 3 Mongo 3.2.13 instances running with journaling enabled, with hundreds of DBs and collections which are accessed simultaneously.
We are checking the option to disable journaling, to improve the cluster's performance. According to the docs, WiredTiger performs a checkpoint every 60 seconds, so in a case of crash we should only loose up to 60 seconds of data. We are ok with that.
Are there any other risks / disadvantages for disabling journaling?
Are there any benefits in disabling mongodb journaling besides performance gain?
The only other marginal benefit is if you are using a 32-bit build (which is also not recommended for production use). Since 32-bit builds are limited to ~2Gb of addressable data for memory mapped files, the journal is off by default to allow for more data (with journal enabled, the 32-bit data limit is halved to ~1Gb). In all other cases, the journal is enabled by default. For 64-bit systems, the addressable virtual memory is not a limiting factor for data size.
Any performance gain from disabling the journal is a tradeoff for data & operational risk. I would strongly advise against disabling the journal unless your data is ephemeral (i.e. you don't care about losing it or can easily recreate it from another source of truth).
If you are concerned about possible contention or overhead from disk I/O of journaling, you could always move the journal to a separate volume. NOTE: moving the journal to a separate volume may also change your backup strategy, particularly if you were relying on filesystem snapshots to take a comparatively quick backup without having to fSyncLock() and quiesce your database.
I know journaling provides write durability and crash resiliency. But I can tune syncdelay parameter to make data files flush more frequently, but mongodb docs recommends against this.
Writes to the journal are fast append-only operations. As you've noted, these provide durability and crash resiliency. If MongoDB stops unexpectedly, it can recover from the journal and data will be in a consistent state.
Without the journal, your data is in an unknown state after an unexpected shutdown and you need to validate it by running repair or (in the case of a replica set node) resync from a known-good copy of the data.
What effect would it have on my mongodb setup and why should I not set syncdelay on production setups?
The syncdelay setting controls the frequency where the in-memory view is synced with the on-disk view of data (aka the "background flush interval").
By default, the journalCommitInterval is 100ms and the syncdelay (background flush) happens every 60 seconds.
If you adjust the syncdelay too low, you can end up creating a lot of additional I/O (and reduced performance) because writes to dirty pages can no longer be effectively batched together and the same page will be re-written multiple times. If you adjust the syncdelay higher, you can create I/O spikes (and reduced performance) with a large volume of changes being committed in a single batch.
The syncdelay and journalCommitInterval parameters are available as tuneable settings because in some cases it may make sense to adjust these (for example, to try to temporarily help an under-provisioned I/O system by reducing write spikes). In a healthy production system, it would be best to leave these settings as-is.
For more information, How MongoDB’s Journaling Works is a helpful read.
It doesn't matter if it's a wiredTiger or MMAPv1, they are just storage engines. Whether you lose updates or not depends on write concerns.
Your app communicates with MongoDB server that writes mostly to memory that has a cache of pages which is periodically written and read from the persistent disk. There is also a journal that logs everything.
When your write to the database it writes both to pages and journal simultaneously. Pages will write to the disk depending on memory pressure. By default you don't wait for acknowledgement from the journal because the journal may not write to the disk for a while, which is represented by j=false. And by default w=1 which means it only acknowledges write to the cache pages.
Default: w = 1, j = false - this is fast, but if something happens before the journal has a chance to write to the disk then the data is lost. What's worse is that it will show as if the data has been written/persisted if you rely on w = 1.
w = 1, j = true - in this case you wait for the acknowledgement from the journal that it has written to the disk. This is a lot slower, but ensures the data is persisted.
Best Answer
When running a journal disabled cluster, we observed frequent disk checkouts - once about every 3 seconds. This was confusing, as the docs specifically mention that checkpoints are created every 60 seconds.
After digging around, we found the following comment on mongo's Jira:
This means that when journaling is disabled, and there are multiple threads writing to the DB, checkpoints become frequent and slow down the entire instance.
This, of course, was a sufficient reason for us to turn on journaling again.