VERR_ACCESS_DENIED when trying to add a raw disk to virtual box

I created a virtualbox raw disk pointing to a functioning, booting USB drive with a small linux distro on it.

sudo ./VBoxManage internalcommands createrawvmdk -filename ~/VirtualBox\ VMs/MyTinyLinux/rawdiskonusb.vmdk -rawdisk /dev/disk2 -partitions 2

When I try to add it to a virtualbox VM, I get this error:

Failed to open the disk image file ~/VirtualBox VMs/MyTinyLinux/rawdiskonusb.vmdk.

Permission problem accessing the file for the medium '~/VirtualBox VMs/MyTinyLinux/rawdiskonusb.vmdk' (VERR_ACCESS_DENIED).

Result Code: VBOX_E_FILE_ERROR (0x80BB0004)
Component: MediumWrap
Interface: IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}
Callee: IVirtualBox {0169423f-46b4-cde9-91af-1e9d5b6cd945}
Callee RC: VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)

I saw that the raw disk vmdk file was owned by root. I chown'ed it to my own user.

Still get the same error. I suspect it would work if I ran virtualbox as root, but I really do not want to do that. Is there anyway to make this work?