Journalctl –verify reports corruption


I just noticed this verify option for journalctl and decided to give it a go, it's showing corruption, what might cause that? and what if anything should I do about it? should I investigate further?

journalctl --verify
PASS: /var/log/journal/19184893a1d645c7a43729e79b10a876/user-1000.journal
Invalid object contents at 3733856░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░   0%
File corruption detected at /var/log/journal/19184893a1d645c7a43729e79b10a876/system.journal:3733856 (of 91734016, 4%).
FAIL: /var/log/journal/19184893a1d645c7a43729e79b10a876/system.journal (Bad message)
Invalid object contents at 21575496░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  45%
File corruption detected at /var/log/journal/19184893a1d645c7a43729e79b10a876/system@60e058db556e4de4b256d0b1ff176aa4-0000000000000001-0004e0b436d20aa1.journal:21575496 (of 44052480, 48%).
FAIL: /var/log/journal/19184893a1d645c7a43729e79b10a876/system@60e058db556e4de4b256d0b1ff176aa4-0000000000000001-0004e0b436d20aa1.journal (Bad message)
PASS: /var/log/journal/19184893a1d645c7a43729e79b10a876/user-1000@60e058db556e4de4b256d0b1ff176aa4-0000000000000a91-0004e0b4ff9a949a.journal
PASS: /var/log/journal/19184893a1d645c7a43729e79b10a876/user-1001.journal

Best Answer

Currently, journalctl can detect corrupt logs but has no "fsck" type command to attempt repair. The journald will automatically switch to writing a new "clean" file as soon as it detects the problem, so theoretically data loss should be minimal.

Until there is a file-repairing command, finding the corrupt journal file and removing it is the only cure. You can find more on this in our Fedora mega-thread over making journal-only logging the default:

For tail corruptions your normal journalctl tool will provide you with as much information as is possible to salvage from the file. It will output the last complete log line and then finish. This is pretty close to how good you can get.

Things are different for corruptions in the middle. We have no nice tool for salvaging data from such corruption, but they could be written relatively easily. However, since they are highly unlikely due to the "append-only" model of the journal this hasn't been on our TODO list.

Of course, if you can identify what caused the problem initially and report it, that would be nice.