Space is available inside the database because data has been moved around. Perhaps you have very high levels of page splits, or have recently deleted a large portion of data that had previously caused the data file to grow.
SQL Server does not shrink database files automatically when you've freed up space within them, because the logical assumption is that if you've used that space once, you'll use it again. Autogrow can be an expensive event and unnecessary if you've only freed up space temporarily (what were you able to do with all that free space in the meantime?). For the same reasons, you shouldn't try to temporarily reclaim space, either. Just let SQL Server use the 18 GB of available space as you add more data. If you think you will need more than 18 GB of additional space going forward (in which case, you will need to add file(s) on other disks, or move to a bigger disk).
sp_spaceused (and in turn the UI dialog you're looking at) may return more space than is possible because of synchronization issues in the metadata about your tables/indexes/files. In order to make sure it reflects accurate space, run this:
DBCC UPDATEUSAGE(0);
I have also seen scenarios where rebuilding indexes was required in order to rectify the counts/space, but I haven't seen that specific case since SQL Server 2000.
(I suspect this isn't a simple case where your database already spans multiple disks, or you surely would have mentioned that in the question.)
That all said, the fact that when you shrink the data file expands almost immediately leads me to believe that you are actually using the space, but must also be performing big deletes or updates that are freeing it up (which is when you see 18 GB free). Unfortunately it's impossible for us to see exactly why the data file is expanding and then clearing itself out - perhaps you have transactions where you are truncating / re-populating large tables, performing massive archive operations, etc.
There is no appreciable difference in performance when the underlying hardware is the same, assuming we're talking about a single physical disk.
However, separate logical drives might help you maintain your sanity, and could result in lower physical fragmentation of the SQL-related volumes
Best Answer
I just did something like this with my system databases for a SAN migration and it didn't seem to have any negative impact. Having said that, I tested it in a dev/test environment. Do you have an opportunity to test this in a dev/test environment?
Also, there's not much of a need to reboot. Just stop the services, do the move, and start the services.
One final consideration is that you copy the files appropriately. Managing NTFS permissions issues can be a real pain so use something like
XCOPY(which is what I used) or RoboCopy to ensure that you preserve the integrity of those permissions.