MongoDB WiredTiger compression stats

mongodb

I'm using MongoDB 3.0.4 with WiredTiger storage engine. I was looking at collection stats and I'm seeking explanation on compression stats fields.
What does "page written failed to compress" indicate? Is it an error I should be concerned about with regards to data integrity and why does it happen?

Sample stats output from one of my collections:

"compression" : {
                        "raw compression call failed, no additional data available" : 0,
                        "raw compression call failed, additional data available" : 0,
                        "raw compression call succeeded" : 0,
                        "compressed pages read" : 205883,
                        "compressed pages written" : 306168,
                        "page written failed to compress" : 3147,
                        "page written was too small to compress" : 51992
                }

Best Answer

What does "page written failed to compress" indicate?

WiredTiger's general strategy is to only compress pages on-disk where there is a storage benefit.

The "page written failed to compress" counter is incremented when compression was attempted but didn't result in an on-disk storage saving (so a page was stored uncompressed).

Another metric for pages that are stored uncompressed is "page written was too small to compress". In this case, the size of a page was considered too small to attempt compression.

The commonly expected case is that pages can be stored compressed, which increments the "compressed pages written" metric.

Is it an error I should be concerned about with regards to data integrity and why does it happen?

These storage metrics are informational counters and expected behaviour (i.e. unrelated to data integrity).

There isn't any obvious actionable outcome based on these metrics, but if a collection wasn't getting much overall benefit from compression the counters might provide insight into how often pages aren't compressed (as compared to compressible pages that don't have a significant level of compression).

Related Question