As this questi9on has had over 1k views, it seems it is a common issue. Therefore, I thought I would outline what I did to get it resolved in case others find it useful.
When I first tried to remove the link, I was told the operation was not permitted. Looking at the link, I could see it was owned by 'root' and in the 'wheel' group. I therefore then tried to remove it with sudo i.e.
sudo unlink X11
This failed with the same error. I also tried
sudo rm -rf X11
but this also failed with the same error. I then posted here asking for help and @Danijel-JamesW added a comment with a link to a useful article which provided some background on the new security features initially introduced in the previous version of macOS, but extended in Catalina. One of the things this article highlights is that some applications, like terminals and editors (in my case Emacs), need to have the full disk access privilege. Without this privilege, you will often get operation not permitted errors in unexpected locations/situations. Unfortunately, this was not the issue in my case. My terminal app (iTerm2.app) had the necessary permissions. However, the article did indicate that you could also get around the operation not permitted error by disabling SIP (System Integrity Protection). This was going to be my next move. However, in the end it was not necessary.
Thanks to a comment from @user3439694 I found out that you can boot into recovery mode and use the terminal to delete the file. THis is what I did
- Boot holding down
commmand + R
- Click on
Utilities and select Terminal from the top menu
- Enter
rm -rf /path/to/file/to/remove
- Reboot
This solved my issue. One important thing to note is that working in the recovery terminal is powerful and potentially dangerous. You need to make sure you are deleting what you mean to delete. To be extra safe, instead of immediately issue the rm command, you can use ls -l to make sure you have the correct file or directory and then hit the up arrow to bring back the command from the history, move to the beginning of the line, remove the ls -l and type rm -rf, leaving the path unmodified following the rm -rf. Note also that the -r means recursive and -f means force - very powerful and dangerous. It will essentially remove everything from the point specified in the path downwards i.e. all sub directories and files. So, if you get that path wrong, you may end up deleting much more than you expected to. In my case, the path I needed was
/Volumes/Macintosh\ HD\ -\ Data/Users/tim/Desktop/Relocated\ Items
my login account is 'tim'. the '\ ' are needed to escape the spaces in the path. Most systems will have the path starting with /Volume, but the drive name may differ (i.e. Macintosh HD). the '- Data' is fairly standard.
Anyway, this fixed the issue and those irritating files are now gone and my Desktop folder is clean (for now!).
Best Answer
Went to the genius bar and the genius booted from an external volume and was able to see both Catalina partitions as volumes. From here it was as simple as moving the "Users 1" folder to the trash and emptying the trash. This can most likely be accomplish through target disk mode as well.