It's possible that there are files in some other user's trash. The .Trashes folder at the top of each volume has subfolders for each different user, by user ID number (e.g. user 502's trash is in .Trashes/502).
You can see if it for yourself using a command like this (replace VolumeName with your drive name):
ls -la /Volumes/VolumeName/.Trashes/
total 0
d-wx-wx-wt@ 3 _unknown _unknown 102 10 Feb 18:15 .
drwxrwxrwx@ 21 root wheel 782 13 Feb 14:17 ..
drwx------@ 35 _unknown _unknown 1190 13 Feb 14:18 502
Note: you might get a permissions error from this command, either because the .Trashes folder doesn't allow read access (solve this by adding sudo, e.g. sudo ls -ls ..., and entering your admin password when requested); and/or because of the privacy protections in macOS Mojave (10.14) and later (solve this by granting the Terminal access in System Preferences > Security & Privacy pane > Privacy tab > Full Disk Access category, see here for more details).
As you can see, on my USB disk .Trashes folder there's a sub foder called 502, owned by user ID 502 (for reference, my current user ID is 501). Since this user doesn't exist on my system, I see it as _unknown, and my user can't look inside of it, neither delete it. To look inside that folder we need to do it as administrator (i.e. use sudo).
If you are sure that you want to, you can delete everyone's trash by deleting the entire .Trashes folder with a command like:
sudo rm -R /Volumes/volumeName/.Trashes
Warning: as with anything involving sudo ("do as super user", i.e. system administrator) and rm -R, use this carefully. If you type it wrong, it could have ... unpleasant consequences.
Secure Empty Trash and FileVault are two different methods for protecting data. FileVault encrypts everything on the hard drive. Only someone with an admin password can decrypt them. This includes anything in your Trash. So by default, the files you delete when FileVault is on are safe via encryption. Even if someone recovered them, they'd still need your password to decrypt.
Secure Empty Trash has nothing to do with encryption. The default Empty Trash just deletes pointers to old files and marks the space they were using on the hard drive as free to use in the future. However, the files are still there if someone ran a data recovery tool or until the OS decides to put a new file over them. Secure Empty Trash prevents recovering deleted files by writing data (zeros) over the space the files you're deleting were using. Meaning the files are completely destroyed.
Here's how it breaks down:
FileVault On | Empty Trash (non-secure) | Someone can still recover those files, but they will recover files that are encrypted per FileVault
FileVault On | Secure Empty Trash | No one can recover the files, so it doesn't matter whether they were encrypted or not to begin with.
Do you need to use Secure Empty Trash with FileVault on? I think it's overkill unless you're afraid that someone will recover files and have your password ready for decryption.
Best Answer
A program that is accessing files in the Trash that you're trying to Empty could interfere with attempt to Empty Trash.
The resulting timeout causes the "Stopping…" (which takes forever) alert message that you saw.
You can turn off Time Machine and try again or perhaps reboot (after turning off Time Machine) and see if the trash empties properly before starting up Time Machine or whatever other programs might have files open.