Linux – How to backup NAS to SpiderOak cloud

backuplinuxnasspideroak

I have a Debian Linux based NAS (network attached storage) from Synology that holds movies, music, photos etc…

The NAS is running the latest DSM 6 firmware built on an Intel x64 processor. I would like to setup an automatic backup job to SpiderOak cloud.

Problem is that there is very little information I could find. SpiderOak basically suggests backing up the NAS by mounting it as an external drive on your PC, as you would backup a USB key. This is problematic for several reasons:

  • Only works if your PC is on and on the same network as the NAS
  • Data flows from NAS >> Router >> PC >> Router >> Cloud unnecessarily clogging your LAN and using your PC resources. Ideally it would just be NAS >> Router >> Cloud
  • Typically, several computers share a NAS; setting NAS and PC data as being from the same device will overly complicate matters when the time comes to restore data

If you successfully linked any NAS to SpiderOak, please share how you did it here. Even if it's a different NAS, I bet people will find it helpful. If we get good feedback, I'll bring this resource to the attention of SpiderOak support

Best Answer

To setup SpiderOak 6 on Synology NAS running DSM 6 firmware

Foreword

The steps below were run on a Synology DS216+ running DSM 6.0.2, but would probably work on other NASes running Linux with small modifications. The SpiderOak version used was 6.1.3

I relied heavily on this page and this page as starting points.

Some things to note:

  • These instructions work as of November 2016.

  • Whenever instructions ask you to execute or run a command, they are referring to SSH (more info below). All SSH commands below should be performed as the root user. I suggest trying to understand what each command does before running it.

  • The packages you should install depend on your CPU architecture. You can find out yours by executing dpkg --print-architecture in SSH.
  • If you know why step 26 below fails or how to fix it, please share!

Enable SSH

  1. Login to DSM (your NAS' web dashboard) by going to http:<NAS-IP-ADDRESS>
  2. Within DSM, open Control Panel >> Terminal & SNMP
  3. Check Enable SSH service

SSH into your NAS

  1. Get an SSH client (I used Putty SSH for Windows; Macs have a built-in SSH client: Terminal)
  2. Find your NAS’ LAN IP address and SSH port. The IP is visible within DSM at Control Panel >> Info Center >> Network tab. The SSH port is 22 by default.
  3. Use the IP and port to connect via SSH. Use the DSM login creds.
  4. In SSH execute: sudo -i to become root user. You will need to provide the admin password once more. For details, see this FAQ

Install Entware-ng (opkg package manager)

  1. From SSH, follow these instructions to install the opkg package manager.
    a. To determine the right version to install, run dpkg --print-architecture. For instance, amd64 means you need version x86 64-bit.

    b. The instructions that require you to edit files can be accomplished by using the Vim editor. For example, to edit /etc/rc.local, run vi /etc/rc.local. Here's a quick intro to Vim

    c. For DSM 6 firmware, when the Entware-ng instructions ask you to edit /root/.profile, edit /etc/profile instead

  2. Restart the NAS, log into SSH and run sudo -i to become root user

  3. Run opkg install nano to get an easier text editor than Vim.

Install SpiderOakOne

  1. Go to SpiderOak website to get the URL to the installation package that matches your device. My NAS has a 64-bit processor so I chose Linux Debian 64-bit and my PACKAGE-URL is https://spideroak.com/getbuild?platform=ubuntu&arch=x86_64
  2. Run: cd /root to open the root directory
  3. Run: wget –output-document=spideroak.install.deb <YOUR-PACKAGE-URL> to download and save the package
  4. Once you get notice that spideroak.install.deb has been saved, press Ctr+c to return to the prompt.
  5. Run: dpkg-deb -x spideroak.install.deb SpiderOakONE to extract the downloaded files into the /root/SpiderOakONE directory
  6. Run: rm spideroak.install.deb to delete the downloaded package
  7. Run: mv SpiderOakONE /opt/SpiderOakONE to move program files to /opt
  8. Run: sed -i /opt/SpiderOakONE/usr/bin/SpiderOakONE -e s+/opt/SpiderOakONE+/opt/SpiderOakONE/opt/SpiderOakONE+ This fixes paths in SpiderOak’s startup script to point to the correct path from step 17. To do this manually instead, open /opt/SpiderOakONE/usr/bin/SpiderOakONE and replace all instances of /opt/SpiderOakONE in the file with /opt/SpiderOakONE/opt/SpiderOakONE
  9. If you plan to backup a lot of data, the SpiderOak data folder will grow large during backups. In my case this filled the system partition and stopped the NAS from working properly. To fix this, I moved the data folder to the general partition where all my data is stored: /volume1

    a. Run: mkdir /volume1/SpiderOakONE to create a new data directory. Replace /volume1 with whatever your general data partition is.

    b. Run: ln -s /volume1/SpiderOakONE/ /root/.config/SpiderOakONE to create a symlink so that accessing the old /root location will be automatically redirect to the new

Add NAS to SpiderOak Account

  1. Run: ln -s /opt/SpiderOakONE/usr/bin/SpiderOakONE /opt/bin/spideroak to create a symlink redirecting /opt/bin/spideroak to the program's real path. This allows us to run the program by simply entering spideroak in SSH instead of having to type the full path.
  2. Run: spideroak --help to see the options available. If that command returns “command not found” step 20 did not work as intended. You could instead run the program using its full path: /opt/SpiderOakONE/usr/bin/SpiderOakONE –-help.
  3. Run: spideroak --setup=- to register your NAS with the service. You’ll need your SpiderOak username and password

Basic commands

  1. Run: spideroak --include-dir=<PATH> to add a directory to the backup set. Example of path might be /volume1/Pictures. --exclude-dir=<PATH> can be used instead to exclude certain directories. If the path includes spaces, you must wrap it in double-quotes.
  2. To see what is currently selected for backup, run spideroak --selection; run spideroak --reset-selection to reset.
  3. To run a backup job, run: spideroak --batchmode You can add --redirect=<FILE_PATH> argument if you wish to generate a log file. The backup will be interrupted if your SSH connection is broken (eg: your PC goes on standby) so I suggest scheduling backups to be executed as cron jobs by the NAS itself (see next section)

Setup a backup schedule

  1. DSM (web control panel) includes the ability to schedule tasks to be run regularly in Control Panel >> Task Scheduler. We should be able to use this to schedule regular backups with a command such as spideroak --batchmode but even though this command works when running SSH as root, it fails when Task Scheduler tries to run it as root. I don’t know why. Instead I edited the crontab (the schedule) manually through SSH.
  2. Look around for how to edit the crontab on your NAS. On mine, the following 4 steps apply:
  3. System scheduled tasks are stored in the file /etc/crontab. Run cp /etc/crontab /etc/crontab.bak to create a backup copy. If something goes wrong with the crontab edits later, restore from the working backup with cp /etc/crontab.bak /etc/crontab
  4. Run: nano /etc/crontab to open the schedule for editing. Nano text editor was installed in step 10. A quick web search will give tips on how to format crontab. For example to run a backup job at 10:30pm each night, add this line to the file:

30 22 * * * root spideroak --batchmode

Columns must be separated by tabs, not spaces. If the command fails to execute, replace spideroak with its full path (see step 21). To log the results, add the argument --redirect=<FILEPATH> and Spideroak will write a log at that path.

  1. If you want to just test whether things work without having to wait for a lengthy backup to complete, replace --batchmode with a much shorter process such as --selection and check the log for the result
  2. Finally, check whether your edits survive a system restart. Restart the NAS, log into SSH, run sudo -i to become root, and nano /etc/crontab to inspect the schedule.

You've done it!

Troubleshooting & Tips

  1. If in SSH, you get the error “command not found” for a command that used to work, it may be because /opt/bin or /opt/sbin are not in the PATH:

    a. Run vi /etc/profile and edit the PATH section of the file (add :/opt/bin:opt/sbin at the end of the PATH string)

    b. Restart the NAS

  2. If you run SpiderOak and get “Command line arguments not allowed during New User Setup”, it’s likely because the symlink we created in step 19.b was destroyed, so Spideroak can’t find its data folder. Redo 19.b.
  3. If your installation gets corrupted, you can re-initialize your device to the server with spideroak --setup=-. This may take a while because after setup the program will immediately complete a backup if needed: took my NAS 12hrs.
  4. I had to factory-reset the NAS once, and so this setup was partially lost. To get things working after a reset, I had to redo step 8.b, 19.b, 20, reboot, and re-create the cron job (steps 26 to 31)
  5. If an SSH command fails with an error about permissions, execute whoami. If the result doesn't say "root", run sudo -i to become root (you will need your DSM password)
Related Question