I want to write a lftp script that will download files every 15 minutes from a server every x amount of time.
Can someone advise how i can do this?
Thanks
ftplftpscripting
I want to write a lftp script that will download files every 15 minutes from a server every x amount of time.
Can someone advise how i can do this?
Thanks
Best Answer
First: Create a script. You can call it whatever you want. I will call it
downloader.sh
.Second: Add it to crontab. If you want to execute it every exact 15 minutes inside an hour:
If you want to execute it each 15 minutes no matter what is the starting minute:
Explaining the variables:
PROTOCOL
- What protocol to use.lftp
supports a good range of them:ftp, ftps, http, https, hftp, fish, sftp
andfile
. https and ftps require lftp to be compiled with OpenSSL or GNU TLS support.URL
- Name or IP of the server. You can even add:PORT
at the end if your server doesn't use the default port of the protocol being used.LOCALDIR
- Where to save the files.REMOTEDIR
- Where tocd
on the remote server to get the files.USER
andPASSWORD
- ftp credentials.REGEX
- Regular expression to filter files to download. It can be useful if you want to download only files of a determined extension, for example. Use*
if you want to download everything.LOG
- Logfile location.Explaining some code logic:
1. - if
The
$?
variable is a special bash variable that means "status code of last command". Bash always return zero on successful command executions so, comparing-eq
(equal to) with the starting!
(negative) on anif
should be enough to see ifcd
andlftp
had issues during execution. If you want a better log of what happened, you will have to crawl through those commands' documentation.2. - heredocs
bash heredocs. It's a way to say "feed this command with this input list". I've named the limit string
DOWNLOAD
so, everything between<<- DOWNLOAD
andDOWNLOAD
will be input tolftp
. You will see examples on the internet with<<
symbol but I prefer the<<-
version since it supports indentation.3. - lftp commands
These are internal commands of
lftp
that means respectively, auth the user with$USER
login and"$PASS"
password, change to$REMOTEDIR
and bulk download anything with the$REGEX
keywords. You can learn them by simply typinglftp
, and as soon as an lftp shell is opened, type?
and press Enter or? lftp-command-you-want
and press Enter. Example:The knowledge related to know that
mget
would be the right command insidelftp
came from reading manpages and searching for keywords like "bulk", "multi" or "mass", and knowing that theftp(1)
command also have themget
command so, probablylftp
would have an equivalent.Manpage:
lftp(1)