After some trial and errors, I've been able to format an exFAT partition on Ubuntu that is working on Windows 10. The trick is to add the msftdata flag on the partition (regardless if partition is aligned or not):

 sudo parted <DEVICE> set <PARTITION_NUMBER> msftdata on

For example:

 sudo parted /dev/sde set 1 msftdata on

And it worked for me in Windows 10!

And in case it helps someone, this is how I created a single aligned exFAT partition in the first place:

 sudo parted /dev/sde mklabel gpt
 sudo parted -a optimal /dev/sde mkpart primary '0%' '100%'
 sudo mkfs.exfat /dev/sde1

exfat should be fine doing so. No read write was happening, the partition does not use journaling

The fact that the file system does not use journaling is the exact opposite of "being fine" removing a drive without unmounting it properly. ;) The OS can't easily fix inconsistencies because there is no such journal.

Use TestDisk

Briefly, your drive still has a partition table and the file system type shows up correctly as exFAT, as seen on your screenshot. Install TestDisk and run it on the device:

sudo apt install testdisk
sudo testdisk /dev/sdb

I have discussed this tool before here on AU, and you might also be interested in the TestDisk guide. As I wrote in this answer on the Unix & Linux Stack Exchange website:

The essential steps are:

• scanning the drive
• selecting the partition
• pressing P to show the files
• copying the deleted (red) files with C

In this case you might want to copy all files, not just the deleted ones.