Figured it out on my own, but WOW is it unintuitive. Here's the solution... follow me on this, it gets a little bumpy.
First, the goal again is to have iTunes (with Apple Music active) show you the complete list of songs for an album where you are missing one or more of the songs.
If you right click a song in the album and select "Go To Album", you will be taken to your local copy of the album (specifically you will see My Music in Album view). Unfortunately this shows you only the songs that you have, and gives no option to display the entire album.
However, if you delete one song from the album (select a track, hit Delete, and confirm that you wish to Delete Item), then the "Show Complete Album" option will suddenly appear.
But, there's a catch. Deleting the item that is currently selected, means there is no selection once the deletion is completed, and so you will be dumped out of the album you were looking at, and taken to the top of your Albums list. Whatever album you were looking at, is now down below somewhere... and you're going to have to find it manually.
Once you again open your album, you are presented with all your existing tracks as well as all missing tracks, which will have an "Add" button next to them. There is also an "Add All Tracks" button.
DO NOT PRESS "ADD ALL TRACKS"! Doing so will add a second copy of all the album's tracks, in your music library. You must click the individual Add buttons next to the missing tracks you want to add again.
But there's another catch. Sometimes - but not always in my case - clicking Add causes the missing tracks to all disappear. I had to "close" the album by clicking on it, then open it again, and the remaining missing tracks once more appeared. Rinse & repeat for every click of the Add button.
But sometimes, it just works fine and you can click Add several times in a row for all missing tracks, and you're all set.
So that's it. Once you see "No additional songs found" and there are no longer any Add buttons, then you've got the complete album once again.
What a huge pain, and very confusing to figure it out. I pretty much got lucky in connecting the dots to figure this out.
Best Answer
No, previous versions of the AppleTV do not support the new Apple Music service. In order to use the service with AppleTV, you'll need to either 1) stream from a compatible device, such as an iPhone / iPad / Mac, or 2) upgrade to the latest 4th generation AppleTV.