According to the Apple documentation about using TimeMachine and Migration Assistant you can select all, several or no User accounts to transfer; select all or no applications; select computer settings and sub-settings or none at all; and finally, other files and folders not explicitly chosen from above.
Additionally, under the User account migration you can chose to migrate
a subset of the Users' account folders, namely any mix of the folders Desktop, Document, Downloads, Movies, Music, Pictures, Public (if it exists), Sites (if it exists) and other files that may reside in the Users's account but not in the Apple default folders mentioned above.
Furthermore, there are arguments for and against migrating the contents of the Applications folder. Some power-users with full sets of installation CDs and DVDs, with all the documentation and serial numbers for the software they own, say it is best not to migrate the Applications folder and instead reinstall all non-Apple software titles from scratch, and to download new copies of other software. While this scheme may be time-consuming and cumbersome, it does guarantee fresh installs. Given the utility of Apple's App store being able to download up-to-date versions of software you have purchased through the store, this may not be as difficult as it seems. On the other hand, if you have a slew of software titles for which you no longer have the distribution media, or for software which is no longer available, migrating the whole Applications folder is your best bet.
Lastly, you can access the TimeMachine disk itself and copy over files & folders manually.
Your point about the restore from Time Machine leveling the effect of the clean install is correct. Unless you restrict yourself to just copying data files alone from a backup, while excluding custom application installs, application settings and system settings, a clean install would only mean spending more time with no apparent benefit.
As I mentioned in this answer to Is it better to upgrade to Mavericks or to perform a clean install?, a clean install is a tedious process that's better done only if you're having some issues in the system that you're unable to trace and resolve through other means.
Discounting bugs in OS X Mavericks itself, you would not notice any stability issues by going the upgrade path.
As a historical note, Apple used to provide an "Archive and upgrade" process long ago where the previous OS would be archived during the upgrade (this was during the time of Mac OS X Snow Leopard and before) and then moved to a straightforward upgrade process that has worked well for several years now.
Best Answer
Restoring an entire Time Machine backup is certainly possible after cleanly installing Yosemite. Just boot to the Recovery HD and choose restore from backup.