Where do open file handles go when they die

The file descriptor 1 translates to the stdout FILE structure in the Kernel's Open Files Table.

This is a misunderstanding. The kernel's file table has nothing whatsoever to do with user-space file structures.

In any event, the kernel has two levels of indirection. There is the internal structure that represents the file itself, which is reference counted. There is an "open file description" that is reference counted. And then there is the file handle, which is not reference counted. The file structure points the way to the inode itself. The open file description contains things like the open mode and file pointer.

When you call close, you always close the file handle. When a file handle is closed, the reference count on its open file description is decremented. If it goes to zero, the open file description is also released and the reference count on the file itself is decremented. Only if that goes to zero is the kernel's file structure freed.

There is no chance for one process to release a resource another process is using because shared resources are reference counted.

I think that most of your questions can be answered simply by remembering that in Btrfs, a snapshot is not really special, it's just a Btrfs subvolume. It just happens that when it's created, it has initial contents instead of being empty, and the storage space for those initial contents is shared with whatever subvolume the snapshot came from.

A snapshot is just like a (full) copy, except it's more economical because of the shared storage.

1. If I don't do snapshots, can you roll back a single file to several changes ago?

No. Just like with any regular filesystem, modifying files is destructive. You can't magically go back to an earlier version.

2. Can btrfs snapshots of root be used and thought of just like VMware/VirtualBox snapshots?

VM disk images are usually block devices, not filesystems or files on filesystems, so I think it's a little different.

You could use a Btrfs file as backing store for a VM virtual block device, I guess. In which case the answer to that question is yes. Except if you use the NOCOW option (which is actually recommended for disk images). Then probably not, because copy-on-write is the magic that makes snapshots work.

3. I label snapshot A, make changes and label it B. If I go back to snapshot A and make changes (even just by booting changing /var/log), are those changes made in a "detached" or "unlabeled" snapshot, so those changes would be invisible if going back to B?

Every subvolume (including snapshots) in Btrfs has a name, so you cannot have an "unlabeled" snapshot.

In general, any changes you make in one Btrfs subvolume (whether it was created as a snapshot or not) are absolutely not ever visible in another Btrfs subvolume. Just remember that a snapshot is just like a copy, but more economical.

4. When deleting a file, is there "this file is deleted" metadata written, so space is still taken by all the versions of the file?

When deleting a file, its directory entry is removed. That is a modification to the directory, and like all modifications, it will be private to the subvolume in which it occurred. Then after that, if and only if the storage space for the file is not used by any other part of the filesystem, it's freed.

Deleting a file whose storage is shared between multiple snapshots is a lot like deleting a file in any regular filesystem when it has multiple (hard) links. The storage [inode] is freed iff it is not referenced anymore.

5. If I build gcc from source, as an example, I think the build directory winds up being 5-8GB. If I build it periodically from source, I'm "chewing up" a bunch of hard drive space, right?

If you build gcc multiple times in multiple different directories, then yeah, it will use more and more space. If you delete copies in between builds or overwrite the same build directory each time, then, no, there's no particular reason why it would keep using more and more space.