Dropping a table does not delete the directories on the disk

I initially just wanted to point at this as a comment, but my rep is too low. So excuse me if this is not a fully fledged answer. (Anyone feel free to edit or hint in comments)

Looking at the documentation at datastax it appears that data will be evently divided by nodes, depending on the "automagical" tokens. From my understanding it's simple as that a new vnode with new tokens just takes an even portion of each node, like all the other nodes already do. Hell, this is even what the doc says basically word by word:

Rebalancing a cluster is no longer necessary when adding or removing nodes. When a node joins the cluster, it assumes responsibility for an even portion of data from the other nodes in the cluster. If a node fails, the load is spread evenly across other nodes in the cluster.

And to address your question:

To further specify my question for virtual nodes does the cluster re-evaluate tokens for all nodes just because a new host has been added?

Sort of. The new node itself takes some partitions (evenly) from all other nodes. If you remove a node every other node will rebalance its partitions from the other nodes, making up for the lost node.

After some tinkering I figured it out; posting here to close the topic and to help others.

All the scripts in the conf/ directory work in concert and there are a number of environment variables you need to redirect if you install in a non-standard place (i.e. a tarball installation).

For each node in our cluster, we're (now) using: (1) cassandra-env.sh (2) cassandra-rackdc.properties (3) cassandra.yaml (4) jvm.options

All the other configuration files are pointed back to the conf/ directory from the tarball using an environment variable.

(2) and (3) set the properties of the node, such as its name, turning on gossip, etc:

(1) and (4) are used by the cassandra bash script when it launches -- the script goes looking for additional configuration files. We've set it so that the cassandra.in.sh is found in the install area; but previously (1) and (4) were not being found, and these are required to activate JMX, port 7199, which nodetool uses to connect to the cluster.