Pool Multiple Hard Drives in ONE Network Share

mountNetworkosx-serverzfs

I've researched for a few days now and could not get any result.
Maybe there is a good tip for Mac I have overlooked.

What I want to do (in a nutshell) is to create a Virtual Folder that contains all of my Disks.

For example I have:

  • /Volumes/Music (1 Hard Drive for Music)
  • /Volumes/Movies (1 Hard Drive for Movies)
  • /Volumes/Movies2 (2nd Hard Drive for Movies)
  • /Volumes/Data (1 hard Drive for Data)

and so on.

Now lets say I create a folder /share and start creating symlinks. Those symlinks are not working, when sharing the folder. Also, when using ZFS, I can only share a whole pool, but if a filesystem is on top of the pool, it disappears.

For example: 

zpool create tank /dev/disk1 dev/disk2 #(merging Movies for example)
zfs create /tank/MoviesA
zfs create /tank/MoviesB

Sharing /tank results in an empty folder, even if MoviesA and MoviesB are filled with Movies.

I also tried to simply share the hidden folder /Volumes but as you can guess, it does not work. I know this does not work and has its reasons, but I tried it to see what is possible.

So the main idea should be clear: creating a network share that contains all drives, but keeps them independent from each other.

I could share a whole zpool, but if a disk fails, it gets lost. Also i could use something like mhddfs but in that case I have no control over the place where a file is stored.

In an ideal situation I would have one big network share called "Server"
which contains all my hard drives by their names, and lets me browse and move stuff from one to another. Simply as said above when I tried to share the folder "Volumes".

I hope there is anybody out there who can help me out with this. There has to be a simple solution…anything virtual.

What obviously is working:

Transmit — Mount the SSH connection under /Volumes

Transmit then creates a Virtual Drive, but this is not shareable via SMB or AFP and does not show the correct size.

What I have not tried yet:
– play around with NFS
– OSXFuse and SSH (probably the same as Transmit does)

Any help would be greatly appreciated.
I do know that there are other options. I also thought about Spans, JBODs, RAIDz, RAID1 Mirror sets in ZFS Setups…

Some Info on the available HDDs: 2×4 TB, 3×2 TB, 1×1 TB and a 640 GB Disk.

If you have any other idea, please share. I am simply trying to have everything in a nutshell, but not like a JBOD so if one drive fails everything gets lost. And I don't need a RAID because the data is not that important and is going to be backed up on the 2×4 TB or the other way around.

Best Answer

There are two ways to do this. I've my combining both solutions I've found that I've been able to maximize both resiliency and flexibility.

1 - Server Side.

Use OSXFuse (https://osxfuse.github.io/) and mhddfs, the latter of which you can install via MacPorts, to create a large drive pool. Then share that. (A wrinkle here here is that mhddfs runs in user space, typically after login, so file-sharing may need to be restarted after mhddfs fires. I'm still trying to work out that kink. If you figure anything out, please do share.)

Know that you can then also share the drives independently. Use the DrivePool share for reading, and the individual shares for writing (so that you have more control over where the files go).

2 - Client Side.

Use auto_mounter and mount all the drives individually on the client, but in the same folder. Although they are individually mounted, auto_mounter is smart enough that after a short sping-up-and-connect delay they basically act as a large, laggy folder on the client machine. This solution is suprisingly robust.

The settings will be a combination of two client-side manifest files, (1) /ect/auto_master, and (2) conventionally at /etc/auto_resources. The best tutorial for this is here - http://useyourloaf.com/blog/2011/01/24/using-the-mac-os-x-automounter.html. It's old, but the instructions still work perfectly on my machine using 10.9.5 on the client and server.

3 - Combine both solutions!

I have my mhddfs master drive pool on my client at /mnt/DrivePool, and the individual drives at /mnt/IndividualDrives/D01 through /mnt/IndividualDrives/D10. It looks a little clunky at first, but it is actually quite robust and flexible for my needs.

Related Question