How to create an APFS-formated volume on an external drive using Terminal

apfscommand line

I want to create a case-sensitive encrypted APFS formatted Volume on my external drive using only the CLI. I had a look at man diskutil already but this doesn't seem to have all the details required. So I'm kind of stuck right now, any help appreciated.

Best Answer

Use newfs_apfs(8) to format a partition with a new APFS filesystem, i.e. to create a new APFS container. For example:

sudo newfs_apfs -e -v "My APFS Container" disk0s2

With diskutil, I find that just entering the diskutil command itself and reading the help that it outputs is more accessible than the manpage. You can do this for subcommands, too; eraseVolume is the subcommand you're interested in, and you can enter diskutil eraseVolume to get help on how to use this subcommand.

The eraseVolume subcommand will overwrite a partiton with a particular filesystem — it will use the relevant newfs_* command behind the scenes, thus serving as a general interface to all of the newfs_* commands — or leave the partition unformatted. diskutil listFilesystems will show you the valid filesystems you can specify. For example:

diskutil eraseVolume APFSX "My APFS Container" disk0s2

Alternatively, if you wish to reformat a partition that has an existing filesystem, keeping the same filesystem type and filesystem label / volume name, you can use the reformat subcommand instead. For example, if disk0s2 is already a case-sensitive APFS container with the name My APFS Container that may already contain data or is broken, and you just want a clean slate, then the above eraseVolume command is equivalent to:

diskutil reformat disk0s2

(Note that diskutil needn't be run with sudo.)

Warning this involves a total reinstallation of your OS, wiping and partitioning your drives. With an SSD, it might be possible to push files into a new partition and by doing this repeatedly, squeeze your data into 3 partitions and then down to the desired two without wiping anything (maybe), but with billions of read/writes going over and over the same cells, such a method has the potential to noticeably age your SSD max life cycles. Backup, Partition and Restore has a much better outcome

My set up: iMac late 2012: macOs 10.14, 1TB HD, 1TB Time Machine, 0.6TB external HD, and about 45 GB synced in 315 GB in GDrive space.

All my personal documents are live synced to gdrive and also backed up to Time Machine. Some of my lesser projects are also synced to gdrive and also backed up to Time Machine. I would strongly recommend you use multiple backup strategies for your most critical documents. I only have a few files in my home directory, rather I have everything linked from the cloud in the Documents, Pictures, Music &etc directories. All my big projects, archives, and video editing I keep in …/Shared as they are way too large to sync to the cloud.

Having said that, I followed the recommendations from Apple and other reliable gurus and accepted that APFS case sensitive is not recommend for applications. So I prepared to split my hard drive 100 GB for macOS ( / ) APFS regular and 900GB APFS case sensitive for /Volumes/Users.

I was going to relocate my home directory and the Shared directory from /Users to /Volumes/Users/……

Determine how much you are going to need for your macOS drive, which mounts at /. First determine how much you have in /Users and the subtract that amount from the total usage on your 1TB. On my iMac, for 110+ apps all the libraries and system files and other normally hidden folders it was about 60GB total, so a 40% margin for growth was fine. Your situation may be different.

On my 0.6TB external drive I have kept a repository of all the dmg, zip, tar, and pkg files that installed programs on my Mac. I have also kept a text file list of all the application in my /Applications and under /usr/local/Cellar & …Caskroom to aid in reinstallations where needed.

Now with a full Time Machine backup of the entire SSD and extra backups, installation files, and lists, you are ready for a smooth transition. And you are covered pretty much if everything goes south too.

With an active internet connection, do the reboot Cmd-Opt-R iNet recover. Do a new install and partition the SSD as desired, first partition APFS regular 10-20% of space (as you have pre-determined will be needed with growth room included) Make a second partition using all the remaining space (less if you have a plan for a 3rd partition for special use) using APFS case sensitive and name it "Users". I would choose that name so as to reduce some possible confusion for certain programs looking for a relative path, but you could call it Data, if you wanted. Continue with the installation and set the first partition (I named mine "macOS") as the install destination drive.

When installing use the same username/password for yourself, this will greatly simplify your efforts.

After installation and rebooting login as yourself and then, if you don't know how to set up root as a user and login as root, create second user, anyname, as administrator and log in as them. COPY the nearly empty new folders from /Users over to the new volume, /Volumes/Users. Choose copy over move because in a worst case loss of your Data volume, you can still boot in as you with your permission intact. (Hint: once in awhile you might want to copy your hidden user setting file over to /Users/yourname—I use midnight-commander to do that. By using a second administrator, all of the files will be free to copy the can be copied. Your admin identity is now safely in two locations. You can logout and back in as yourself.

You are now ready to move your home folder assignment. In the System Preferences/Users & Groups, unlock the settings first. Remove the second admin you created and all their files & directories. Now right-click on your ID in the left-hand table and choose advanced options. It is here you can "move" your home directory to the new one on the second, case sensitive partition, /Volumes/Users/yourname.

Once you have changed your home directory, you can reboot and and log in. /Users/yourname and …/Shared are now orphaned.

Here is where you have to choose, speed OR security and reliability: Note you can not just restore as that will try to put everything back on the now much smaller first partition.

A. the faster method:

open two finder windows, one at / the other at Time Machine
press command-shift-. to see all the hidden directories
use finder to navigate to the last full backup on Time Machine
highlight everything in that backup EXCEPT /Users and copy all the folders over to root ( / )

This method holds the probable risk that your moved identity will not recognize most of the restored programs/have the right settings for them, and if you then go on to restore yourname from the Time Machine in the same way, you also won't have the right settings restored.

B. the surer (slower) method

Next, you should re-install all your previous programs. Once done, you should re–establish your email settings and other on-line accounts through System Preferences. If you archive your emails, like I do, you can't just restore them because the directories are no longer have the same names. However, you can import them through Mail out of Time Machine under your old file structure, ~/Library/Mail/ into your "new" email. Once they are imported you can rearrange them as you like.

Important: do not re-establish your Time Machine drive as the current backup drive before you are satisfied that everything has migrated correctly, or the app will start dumping older files to make room for the "new" ones without ask you.

Next, you can start restoring all your Document, pictures and such, and if you have used …/Shared to file all your project data, and however you have used it, those folders can all be dragged out of Time Machine and copied back to their original location. I recommend this way as it will make sure that all of your permissions remain as before.

When finish, your OS should be on a APFS case-insensitive boot partition and all your user identity and all your data should be on a APFS case-sensitive data partition.

You can go back into the Time Machine if you find some settings just won't work and look for the originals.

If all else fails, you can rewind it all back by restoring (with re-partitioning) from your original Time Machine backup.

I was up and running in about 4 hours but I have a LOT of customizations and it was three more days before I was happy.

Be patient and methodical and this should work well for you.

Richard

Carbon Copy Cloner – Create Encrypted Bootable HFS+ Clones from APFS Source

Here's what worked

Download the High Sierra installer from the App Store.
Run the installer, choosing the external clone disk as the target. This will automatically create a new Recovery HD partition associated with the target disk.
Reboot from the external disk. Hold down option key during startup to select the boot disk.
Enable FileVault encryption as usual from the Security & Privacy panel in System Preferences.

Additional Notes

You can reboot from your internal drive for better performance, after enabling FileVault on the external. The encryption task will continue as long as you're plugged into power and the external disk is connected.
It may take several hours, or even days, before encryption is done, depending on the volume of data to be encrypted.
You can view encryption task progress using the Terminal command diskutil cs list

What didn't work

There are some articles posted about how to clone the internal disk Recovery HD volume to any other drive, but I didn't have success with them:

$ diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.3 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                 Apple_APFS Container disk1         499.4 GB   disk0s2

/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +499.4 GB   disk1
                                 Physical Store disk0s2
   1:                APFS Volume Mac-HD                  431.9 GB   disk1s1
   2:                APFS Volume Preboot                 22.9 MB    disk1s2
   3:                APFS Volume Recovery                518.8 MB   disk1s3
   4:                APFS Volume VM                      2.1 GB     disk1s4

$ sudo hdiutil create ~/Desktop/Recovery_HD.dmg -srcdevice /dev/disk1s3
hdiutil: create failed - No such file or directory

I didn't find any articles that specifically addressed the case where the source disk is APFS format.

Best Answer

Related Solutions

Change APFS Volume to case sensitive

A. the faster method:

B. the surer (slower) method

Important: do not re-establish your Time Machine drive as the current backup drive before you are satisfied that everything has migrated correctly, or the app will start dumping older files to make room for the "new" ones without ask you.

Carbon Copy Cloner – Create Encrypted Bootable HFS+ Clones from APFS Source

Related Question