RSYNC failing randomly with rsync: mkstemp failed: No such file or directory

rsync

I use rsync to send files to a remote server mostly using a limited bandwidth connection that can also have packet loss or jitter, sometimes randomly rsync will fail with the following error:

default_perms_for_dir: sys_acl_get_file(photos/2019-05-11_01-31-51, ACL_TYPE_DEFAULT): No such file or directory, falling back on umask
rsync: mkstemp "/photos/2019-05-11_01-31-51/.data.txt.CJXeor" (in remote_bkp_site) failed: No such file or directory (2)
rsync: mkstemp "/photos/2019-05-11_01-31-51/.output_20190511-013202.mp4.Bhka2P" (in remote_bkp_site) failed: No such file or directory (2)

If I run the rsync again it will work and successfully send all the necessary files to the remote site.
This error happens randomly, not on the same file.

I use rsyncd daemon on the server side and my root directory may have about 1000 files. The old files doesn't change, I mostly use rsync to push new files to the server.

I did a lot of research and found out this error mostly happen to mounts ( CIFS ) or permission error but it's not my case. Also if I run rsync again a couple of times it will eventually sync all the files.

I believe this error has to do with the fact I sync the servers over a 4G broadband connection that has a limited bandwidth and packet loss can occur.

Do anyone has any suggestion or encounter a similar situation ?

Best Answer

I have found under linux that when transferring files across different fs types, the mkstemp call can fail due to a valid name on the source system being invalid on the target system.

This means even a touch of the filename will fail & not all fs seem to accept escape sequences to avoid this. I verify that I can touch into the target folder, but it fails when I incorporate the first invalid character (eg, ':') even when escaped.

Related Question