My current storage (RAID-1 off of a hardware RAID card) and backup (a spare drive) solutions for my home network are inadequate. I have too much data scattered on various one-off drives. It is time to evolve. Backups seem simple enough, at least: lots of big drives. However, I am bewildered by the number of choices for small home storage. The Drobo S looks appealing. So does the ReadyNAS. I am not looking for bunches of shiny features, I'm mostly interested in reliability. I am not interested in building Yet Another PC to create a file server or doing something in the cloud, or whatever. I'm stupid, so I am keeping it simple.
Requirements for Main Volume:
- Starting working space roughly 2TB, with options for growth up to 5TB
- RAID or something RAID-like with at
least one parity drive - eSATA II for speed during backups
- Ability to shut down gracefully when
alerted of low power by a UPS - Optional but Desirable: Will take
2TB drives now with options for the
larger 3TB drives coming in
2010-2011 - Optional but Desirable: : RAID-6 or
something similar, with two parity
drives - Optional but Desirable: : Hot spare
- Ethernet connection not required, as
the volume will be shared via the
same machines which runs my home
print server
Backups:
- Backup performed via ROBOCOPY in
mirror mode to an external hard
drive via a eSATA II connection. - Start with rotating between two
external 2TB hard drives, will go up
to six external 2TB drives. - Start with a weekly backup, move to
a bi-weekly backup as more drives
are added. - Move to 3TB drives as the size of my
main volume increases. - Backup drives will be stored on an
off-site location.
Hard drives:
- I plan on buying all of the same
model, but different batches from
different vendors. - I found a "burn-in" utility with
which I can pound away on the drives
for a couple of weeks before adding
them to the backup pool or the main
volume.
I estimate that I am looking at roughly $1,500 to start, once I start throwing in two TB drives for backup and four for storage. So, are there any obvious flaws in my plan? What have I overlooked? Any suggestions for the storage device for my main volume that fits my requirements? Or do I just keep it simple, 2 drives in RAID-1, then perform due diligence with my backups, accepting that I will have to buy a whole new unit when my data grows past 2TB?
Best Answer
Main Volume
I know you said you don't want to build another PC fileserver, but most of the ready-to-go solutions don't have any safeguards against silent data corruption.
If you're looking for data integrity and reliability, you might want to consider running an OpenSolaris fileserver with a raidz2 or raidz3 configuration (2 or 3 parity drives, respectively) on ZFS.
With larger drives, the rebuild time will increase when a drive fails--which also increases the chances of a second-drive failure during the rebuild. But the main advantage of ZFS is that it protects you against silent data corruption, since the filesystem itself is checksummed.
You can also run ZFS on other operating systems, but OpenSolaris is always the most up-do-date version since it takes a while to port new features to the other platforms. If setting up an OpenSolaris box seems a little more work than what you want, FreeNAS seems to be the next best thing, in terms of ZFS support.
On the Linux side, ZFS is not supported in the kernel (only as a user-level driver), but there is also a new filesystem under development, called btrfs. Unfortunately, there is no stable release of btrfs, as of March 2010.
Backup
For your offsite backups, it might be more cost-effective to pay for a service like CrashPlan, Carbonite, or Mozy. It's very, very easy to configure any of these to automatically backup your files. Of the three, CrashPlan has the best backup and recovery features (and even allows you to backup to other remote computers for free), while Mozy's recovery methods are either expensive or very inconvenient (if you want to download a Mozy backup, you have to wait for your job to be queued up and bundled into a zip file). I haven't personally had any experience with Carbonite.
Note that you shouldn't depend solely on an offsite backup--if you backup to the cloud or some other offsite computer, you should also have a local backup.
The Drobo reviews I've seen noted poor write performance, but if you're just using it as a nightly backup drive, it might be sufficient.
Backup rotation
If you want to rotate backups between a local and off-site location, you need at least 3 backups to guarantee one is always local and one is always safe at the off-site location. The third is either in-transit or at one of the other two locations at any given point in time.
ROBOCOPY vs. CrashPlan
ROBOCOPY will cause more wear and tear on your hardware, since it has to read every file during every backup. It's not clear to me whether it only copies changed files or if it copies all files. If ROBOCOPY fails for some reason, it may not be apparent that it has failed, unless you have set something up to reliably report its backup status.
CrashPlan monitors your hard drive for changed files, and only copies the changed files. Since it actively monitors changes to disk, it does not need to read every file in your backup source. CrashPlan automatically e-mails you to notify you how long it has been since the last backup, and how much data was transferred during the last backup.
That said, keep in mind that CrashPlan doesn't have to replace your ROBOCOPY backup scheme. You can use CrashPlan to supplement whichever other backup scheme you happen to choose.