I think you would have to "merge" the layers (right-click). You can drag layers into layer groups if you need to group them. Then you can drag the entire group around the page.
I personally have never dragged a layer onto another layer. I merge them.
Best Answer
Alt click the layer's eye that you want to show and all the other ones will be hidden.
Clicking again with alt will invert the same action.