The symptoms are very consistent with a mostly saturated IO system, however having for the most part ruled out IO load from the OS/userspace side, another possibility is the drive running self-tests on itself, which may include reading from all the sectors. This should be queryable/tunable from smartctl (At least one place being smartctl -c for querying).
As for why it's coming and going and started suddenly now:
- The drive has passed a certain stage in it's life (number of sectors written, time spun up, etc.) and the firmware on the drive have triggered one of these scans
- I believe this also can be triggered via smartctl, so it's possible some automated process triggered it
- Having one of these scans triggered and flagged as either in progress or started, when the drive has spent a certain amount of time powered on, it's re-triggered either from the beginning or to resume where it left off
Is moving file from tmpfs to ext4 atomic?
No. Renames as such only work within a filesystem. The manual page for rename(2) explicitly mentions the error that is returned if trying to rename across mount points:
EXDEV oldpath and newpath are not on the same mounted filesystem.
Moves across file systems need to be done as a combination of a copy and a delete. mv will do this for you if the rename() doesn't work, but it will not be atomic in that case.
The simple way to work around that would indeed be to first copy the file to a temporary location on the same filesystem. In general, it's simplest to place the temporary file in the same directory as the final destination, since that's the only place that's guaranteed to be on the same filesystem.
Of course that requires that any process working on the files there will have some logic to ignore the temporary based on its name.
Roughly, something like this should work for one file:
cp /src/filename /dst/filename.tmp &&
mv /dst/filename.tmp /dst/filename &&
rm /src/filename
Note that the process you describe for a directory is essentially this:
cp -r /src/dir /dst/dir.tmp &&
mv /dst/dir /dst/dir.bak &&
mv /dst/dir.tmp /dst/dir &&
rm -r /dst/dir.bak
Which is not bad, but is not atomic. There's a moment of time between the two runs of mv (or calls to rename()), when /dst/dir does not exist. That could be worked around by accessing the directory through a symlink, since the link can be atomically replaced with a rename.
Best Answer
No, you don't need to run
syncbeforeumount.umountwill complete all pending writes before it actually unmounts the filesystem. It will also refuse to unmount if some process is still using the filesystem, e.g. as current working directory.Edit: Unmounting is mostly handled in
fs/namespace.c. You won't find any explicit call tosyncthere, but you'll see comments along the line of "mark this mountpoint for unmount, refuse any further operations on it, and if all operations are done, unmount". You can also see explicit in-use checks.You can easily test yourself that
umountreally does finish all pending operations: Mount some slow USB stick, copy a large file to it, and directly callumountaftercp. It will take several seconds before you see a new prompt, and if you rundstatetc. in another window, you'll see the write operations that are still going on. That's exactly the same behaviour as if you've typedsync.