IPhone – How to stop from having the music transfer between the iPhone and the HomePod

What initially seemed like a clever feature turns out to be ridiculously annoying: the transfer of a song between a HomePod and an iPhone.

I have a HomePod in my office, on my desk, and whenever I take out my iPhone to do something, even though the iPhone is a good 2 feet away from the HomePod, it transfers from the amazing speaker of the HomePod to the little crappy speaker in the phone. This literally happened like half a dozen times in the last song I played. It significantly degrades the listening experience both by interrupting the song (there is always an extremely noticeable delay when transferring) and by forcing my to play on a radically inferior speaker.

How can I make this stop?

I have looked through the configuration for the HomePod and can't find anything to allow me to turn off this misfeature, or (better yet) adjust the proximity required, or add a confirmation before the transfer. This seems like an obvious failure on the part of Apple to test this feature well or provide necessary settings, but since I can't make them fix it, what can I do?

There is no other furniture in my office that is suitable for the speaker, so moving it is a terrible option.

I tried turning off Bluetooth on my phone, and that surprisingly didn't thwart it. (I would have guessed that Bluetooth was used for detecting proximity.)

When the notification appears to tell me of an impending transfer, flicking it away quickly has occasionally seemed to prevent the transfer but this has been very inconsistent, and there is a very short opportunity to do that.

To add insult to injury, as I've tried testing various methods I've found that the proximity detection often fails even when I want it to work, making it impossible to test whether some behavior is stopping it.

Update: I've found a solution to turning it off completely (below) but I'd still be interested in other approaches if anyone finds a clever way of controlling how this works. The ideal would be a way of having it turned on, but having it work reliably only when I actually intend it to, not when I'm just using my phone from a couple of feet away from the speaker.